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ABSTRACT 

The Formative Evaluation and Heuristic Research 
(FEHR) PRACTICOM, a computerized game, gives practical experience in 
such program evaluation tasks as problem definition, operationalizing 
objectives, field study, budgeting, proposal writing, data analysis^ 
and interpretation — without the expense and time commitments required 
by most research. The players, graduate students or practicing ^ 
professionals, design and conduct an experiment comparing the 
effectiveness of available educational programs on students in a 
mythical school district. As a "research assistant," the computer can 
identify individuals or groups of students by category (test score, 
grade, sex), apply educational programs, and administer and score 
tests, which determine the effects of a program over time. There is 
no predetermined decision; rather, the results of each team are 
critically evaluated and methodological implications are discussed. 
The Game Manager's Manual is a comprehensive guide to FEHR. The 
introduction describes experiences which FEHR can provide, and the 
author's perspective on the program evaluation process. Three 
detailed descriptions are presented: (1) Players* Introduction — an 
overview of the game, its physical components, and des.criptions of 
the problems available, (2) Fair City O.S.A.~a chamber of commerce 
description of the mythical community, and (3) players* step-by-step 
instructions for completing tasRS assigned by the Game Manager. 
General guidelines for setting up and validating the computer 
program, training instructional staff, and adapting FEHR to local 
needs and purposes are included. (CP) 
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SECTION i; 

INTRODUCTION 



FEHR-PRACTICUM is a computerized simulation which proviaes practical 
experience in decision-oriented educational research and evaluation. It 
is intended as a pedagogical tool to facilitate instruction in such program- . 
evaluation tasks as defining the 'problem, operationallzing objectives, de- 
signing a valid field study, budgeting, writing proposals , /analyzing data, 
and interpreting outcomes with respect to an imp'^nding decision. The acro- 
nym^ FEHR (prorvDunced "fair") sta.ids for Formative Evaluation and Heuristic 
Resed-ch. Formative evaluation refers to an assessment during the develop- 
ment of a. program which performs the functions of feedback, diagnosis, and 
guidance. Heuristic research is meant to^^uggest a decision-oriented pro- 
cess that seeks practical solutions to educational prcfclems. The name FEHR- 
PRACTICUM was intended to emphasize our focus on a practical problem-solving 
experience which features the use of research/evaluation technology in making 
decisions aoout educational programs. 

Purpose 

This manual is intanded as a comprehensive guide for the implementation 
and use of the FEHR-PRACTICUM system. In addition to a comprehensive descrip 
tion, it provides detailed instructions for planning and administering a prac 
ticum session. To facilitate instructional planning, a discussicn of the con 
ceptual foundations of the FEHR approach is provided, together with a brief 
list of instructional guidelines for each choice point in the flexible FEtiR 
format. A variety of practical examples are given to illustrate the purposes 
which can oe served. 

Or ganization 

In general, this manual is organized sequentially on a need-ttf-know 
oasis. The broad outlines of the model are presented first, followed by a 
aetailea description of each task as it arises. A complete front to back 
reading of the manual is recoiwnended prior to the first practicum session. 

The manual contains four major sections. A brief oescription of the 
general nature and purpose of the materials in each section is provided 
below. 



Section I. Introduction. 

the remeinder of this introductory section provides en overview of the 
entire FEHR-PRACTICUM system; Itsipurpose It to ecqueinl potential 
users with' the author's perspective on the program-evaluetlon process 
and to describe th^ part which FEHR-PRACTICUM mtflht play tn research/ 
evaluation training'-- both pre-servtce and tfl-servtce. 
Section II. Detail ad Description. 

The second section contains a detailed description of FEHR-PRACTl^, 
suwnary description of the eight problew available In the systemfe . 
list of the tasks involved in a typical practtcuro, and step-by-step In- 
structions for completing each task, f 
Section III, flanning and t^eetnq a Session. 

The third section provides detailed Instructions for planning and manag- 
ing the first FEHR-PRACTICUM session at a new location. The ftrst half 
of the section contains general guldellms for setting up and validat- 
ing the conpuUr program, familiarizing Instructional staff wtth the 
system, and adapting FEHR to the local needs and purposes. Practical 
examples of various uses of FEHR-PRAaiCUM are provUed fellowtng the 
general guidellngs. In the interests of clarl^. all examples and tllus 
trations in this section are based on a standard problem tn remedial 
arithmetic. 



Problem 5 has been deleted from these materials becaus«l of intric 
problems in simulating data at discrete time periods. 



Need 

In late 1969^ education entered an era in which its sources of revenue 
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Degan -to dry up while its costs contined to clitnD at an accelerating rate. 
The combination produced an inexorable demanu for ecitcators to provide evi- 
dence that their programs were, in fact , producing the results for which they 
weVe intended. Simultaneously, educators themselves, faced. with austerity 
budgets, began to clamor for information which would help them decide which 
programs wece most etfective and efficient and. alternatively, which could be 
most easily sacrificed". Many were surprised to discover that personnel who 
could supply relevant, convincing informatics were largely unavailable — 
despite the intensive national research training effort of the sixties. 

The reasons for this appare it failure are discussed in a comprehensive 
report by the Phi JeUa Kappd National Study Commi ttee. on^Evaluation (Stuffle- 
beam et al. . 1971, pp.' 302-307). Col lecti ng* val i d information for this kind 
;0f educational accountability, they' say, reqik'r^ personnel who are skilled " 
in adapting and integrating the idsas an. methods \f classical educational/ 
psychological research, economics, political science, administration, decision 
theory, and general systems theory to meet the specific needs of an impending 
educational decision. 1 ersons with'these skills are hard to find even 
among the graduates of doctoral programs' in educational research/evaluation 
at our most prestigious institutions. Although they identify certain con- 
cepts and techniques which need further development, the PDK Committee points^ 
out Ip?., 307-308) that most major universities currently offer courses which ~^ 
could develop most of the required conceptual skills. What is missing, they 
say is a careful ly-plannea sequence of apprenticeship or practicum experien-^ 
cfes which can be completely integrated with the instructional activities of 
the regular curriculum. 

Instructional Role 

The traditional apprenticeship or practicum experience is unsuited to 
the training task described above for two reasons: CD it is usualjy too far 
removed from the classroom in both time and distance to permit etther direct 
application of the principles s+ .'ied oT planned reinforcement activities, 
and (2) the sequence of activities dictated by the needs of the project sel- 
dom coincide with the instructional objectives of the training program. How- 
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ever, FEHR-PRACTICUM permits students or practicing professionals/ who wish to 
up^irad^ their »ki^ tor gat practical experience in a variety of \ealistic, 
dftcisi on-oriented field studies ranging from the validation of a questionnaire 
for student e'^iOiiatictn of teaching at the college level xo the assess^nt of 
an elemental^ reading program or the evaluation of a Headstart project. Each 
of the above examples is based on a FEHR-PRACTICUM problem. There are eight 
major problems available, each set in a aiffereht content area and involves 
^ subjects at a differemc educational level. 

It it important to understand at the outsk that FEHR-PRACTICUM is not 
intended to provtde ihstruction in research/evaluation techniques. Rather, it 
provides an opportunity to apply theoretical principles to practical educational 
problene, to practice and develop research/evaluation skills i a complex eny 
vironmentHbich requires constant extension, generalization, and adaptation 
of those principles. Pedagogical ly, FEHR-PRACTICUM is a manageable field 
experience wl^ich is always accessible. It provides a safe vehicle for practicing 
complicated research strategies, and it provides imnedi ate feedback on the 
effects of long-term treatments. When carefully articulated with an appropriate 
training pJogram, the practicum can provide a thread of continuity about which 
disparate ideas coalesce, thUs promoting integration ano synthesis. 

yowever, FEHR-PRACTIpUM, like other field experiences is not particularly 
fruitful fin Isolation. If the practicum Is not accompanied by planned instruc- 
tion, it is imperative that the player- trainees have access to expert consult- 
ants and/or ample reference materials designed for independent use. A discus- 
sion of the instructional implications of Integrating fEHR-PRACTICUM into an 
existing training program is provided in a subsequent section. 

Practicum Environment 

An important advantage of a FEHR-PRACTICUM. is that It allows participants 
to try out new approaches to planning, budgeting, and evaluation without sub- 
jecting real student-subjects to the uncertainties of experimental conditions. 
Instead, the subjects for FEHR- PRACTICUM experiments are drawn from the simula- 
ted school system of Fair City, which is located in the mythical state of Uto- . 
pia, U.S.A. The instructional effectiveness of the practicum is directV' re- 
laUd to the quality and depth of this simulated environment, 

/ - . -10- 

10 



Populations Ser^ved 

The format of the FEHR-PRACTICUM problems wa^ specifically designed for . 

-flexibi'lityr Basica>iy, this was accomplishea by providing a checklist of 
optional a^&i,gnnients "which' allow the" game manager to adjust both the scope 
of the practicum as a whole a'na t.ie complexity of each of the tasks involved. 

'Guidelines for choosing the options bes^ suited to the instruc1;jonal purposes 
of a particular* practicum session are provided in section III of this manual. 

This extremely flexible structure of the FEHR problems makes it diffi- 
cult to anticipate all the potential clientele. However, the authors have 
successfully used FEHR-PRAcfltUM as' the core experience, in each of the foll-OW- 
inj'training J'ctivitles: 

V 1 A twelve-session Saturday morning extension cdurse Cworkshop) designed 
to acquaint educational administrators (p. marily linncipalsj w.th 
the basic principles of empirical pi'ogram evaluation. The courses 
emphasized problew conceptualization skills ano' the ability to conmun- 
icate with statistical consultants. 

2 A one-semester laboratory practicum designed 'to acquaint first-year 
graduate students in Special Education with the strengths and weak- 
nesses of various standardized tests commonly used to d\agr,ase Team- 
ing aisabilities, the principles of differential diagnosis, and the 
basic ideas of ' research design and statistical analysis. • 

3 A two-semester sequence of research design and data analysis cour- 
ses required of all Ph.u. students in education. Mos-t of these stu- 
dents had no previous research experience, and mahy had previous y- 
established negative attitudes towards mathematics and were openly 
anxious at the prospect of learning statistics. i' 

4 

Experiences Provided the Game 

FEHR-PRACTICUM is intended tr provide a^wide range of practical exper-» ^ 
ience tn educational research and evaluation with^^out the expenses arid time^ 
conunitiRents involved in reel research. " Th^ practicum provides p-layjers witH di- 
rect experience in gathering and analyzing empirical data in order to arrive 
at a practical educational decision, and provides feedback respecting the .ade- 
quacy of their decisions. 

Given tne goal-of simulating the entire research/evaluation experience, 
common sense would di.ctate that the closer the simulation is to reality, the 
more valuable will be its -contribution to practice. Consequently, a con- , , 
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sclous attanpt Is mad* to provioe many of the complex Interactions (and fus- 
tratlons) whicft are c;iaracteHst4c of field research as opposed to laboratory 
research. A partial list of tJie experiences which can be provided appears be- 
lo*. However, ti»e user has th« option to emphasize one experience and de-em- 
phasize (or omi tj another. Instructors, may choose the combination best suited 
to their needs., FEHft-PRACTICUM has the capacity .to provide practical experience 
in each of the folloMing areas: 

1. Identifying and making explicit tha basic or "r^al" problem. Con- 
fceptually, a "real" problem may be defined as the discrepency between 
what is happening and what should be ha?penino. However, it is a 
cornnorToccHrr^nce in real-life evaTuation work to be presented wtth 

a "problen" which is,' in fact, a request for an implementation 
decision about one of a series of alternate solutions. For exaiiiple, 
"Should we implement program X?" is^a solution masquerading as a 
problem. Identification of the basic problem facilitates the ident- 
ification of relevant dependent (or criterion) vartablts. Note : 
This is perhaps ^:e Host difficult task Ccoi»ceptu«lly) tn theentire 
practicia. Question i of relative wlue and whose valu* system must 
be dealt with. ^ 

2. Stating a problem in operational terms. The practicum provides con- 
siderable practice in this area since the computer requires all re- 
quests for inforaation to be Mdt In tam of the valuas of particu- 
lar variables. 

Preparing a budget and working within its constraints. In all except 
a few restricted versions of problems, the players are given a fin- 
ite research grant and must p^ for each bit of information they 
collect. In aadition, players must pi^y themselves a daily salary. 
Thus, careful 4)lanning of expenditures of both time and money is 
necessary. 

developing and following a samplihg plan. The average FEHR-PRACTI- 
CUM problem contains literally thousands of potential research sub- 
jects, each with a wide variety of individual characteristics (sex, 
intelligence* socio-economic status, etc.). Almost any sampling plan 
which can be use J in real research can be duplicated in the game- 
including plans which are Invalid because of some type of selection 
bias. ~ 

Selecting dependent and Independent (moderator) variables which/.are 
relevant to a giveri problem ana choosing the instruments (tests) 
which will be used- to measure them. Although the players cannot 
devise their own tests, they na^ choose from a large pool of tests 
which are made available in the practicum. To help them in assessing 



3. 
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tfie utility of the various tests, players have access, via^the In- 
formction LariK, to test descriptions of the sort provided by Buros 
(1985). Depending on the problem area, the game provides scores on 
from 50 to 160 separate tests. Each test may be used with any subject, 
ana may be administered repeatedly across time. 

Using survey techniques to identify the important dependent and In- 
dependent variables in a given educational problem. In the practicum, 
surveys are frequently required to detemiine the ftf^t and severity 
of a problem and/or to clarify the relationsh-i variables. 

. Designing research plans which isolate the effects of specific educa- 
tional treatments and treatmeat combinations. The practicum allows 
placers to collect data according to almost any research design which 
can be used in a real-life situation— including biased or invalid 
oesigns. The possible designs include both the univariate and multi- 
variate forms of latin squares, incomplete blocks, longitudinal 
studies (panel data), and case studies based on w>riable scores (ra- 
ther than verbal descriptions). Because of the capacity to produce 
longitudinal data.^it is possible to simulate formative evaluation 
studies involving sequences of treatments and repeated observation 
periods. 

Analyzing data collected 'from complex designs. The capacity to pro- 
vide s'":h analytic experience is ensured by the complex designs men- 
tioned in /7) above. In addition, the game has a number of built-in 
biuses whith encourage plavers to use designs involving multiple cri- 
teria (dependent variables). Thus, multivariate analyses are us- 
ually appropriate. (Of course, the capacity av-tualTy to conduct 
such analyses depends on the resources of the local computer install- 
ation.) 

9 ■■ Modifying research plans to accor.modate unforeseen events in the en- 
vironment. For example, a teacher strike could modify student atti- 
tudes as well as cause an expensive delay in a project. Such simu- 
lated events can be used with sophisticated trainees, but are not 
rei,jmmended for beginners. 

0 Selecting consultants' and preparing plans to optimize tLeir effective- 
' ness. The practicum provides an opportunity for players to explore 

their , own limitations, and to find the conditions under which a con- 
■ sultant is "worth the money." 

1 Kelating the results of an evaluation to the time at which the eval- 
uation i« taken. -The game permits program evaluations to be made 

at different points in time and to compare the results. 

2 Working with education4 problems in a variety of content areas and 
at numerous educational levels. The topics of the e'.ght problems 

^ available run from the traditional subjects (e.g., mathematics and 
reading) to the specialized difficulties of handicapped children. 

■ -13- 



The educational levels represented include both pre-school children, 
and college students. 

Assessing the quality of research procedures (in the comparative phase) 
by examining the results obtained from "operating" a -decision based 
on the research results. To aid in this task, the computer supplies 
two bits of information which' are not obtainable in real -life re- 
search: the "decision effectiveness index" and a statistical summary 
of the characteristics of the students best suited to each treatment. 
These are not intended as absolute indices of quality, but rather as 
springboards for discussion. 
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PLAYERS' INTRODUCTION: Page 1 



PLAYKRS- INTRODUCTION TO THE FEHR-PRACru-lIM .-.Ml 



Program research and evaluation have always bcoii i.npori.uu .utUitu-s i„ 
education, but they have recently become even more important bcc^mse of a 
growing public emphasis on the twin themes of innovati on' and accountability. 
On the one hand, there is mounting pressure for hold new educational programs 
to meet the chamdng needs of an increasingly urbanized society, anH on the 
other, an accelerating demand for educators to prcide evidence that their, 
programs are in fact producing the results for which they were intended. 

You are about to play the FEHR-PRACTICUM game, which was designed to 
familiarize you with the problems and complexities of conducting program re- 
search and evaluation studies in educational systems, such as the public 
schools, and to provide an opportunity for you to practice your research/ 
evaluation skills. The name FEHR was chosen because the game emphasizes 
formative evaluation and heuristic research; that is, evaluation conducted 
to provide feed^ack and guidance during program development, and research 
projects intended to provide information to facilitate specific decisions. 
Playing the game will give you direct laboratory experience with planning 
and budgeting a research/eval aation 
project and with analyzing and in- 
terpreting the results. However, 
FEHR-PRACTICUM projects are simu- 
lated rather than real research 
because they are conducted in a 
simulated educational system which 
exists in the mythical community 
of ] air cMty , U.S.A. 




PLAYERS' IWTRODUCTION: Page 2 
What Are the Advantages of Simulated Research? 



If indeed, " experience is the best teacher " and " wo learn by doing", 
it would seem that a sound educational course is to allow people to be directly 
involved in th« situations in which they express interest. Thus, we would have 
aspiring plumbers plumb, surgeons operate, and educational researchers research. 
However, learning research and evaluation skills by doing real field studies 
presents sfone serious practical problems: 

(1) Frequently, a trainee's initial efforts will be failures. 
In education, this might- involve an unacceptable risk to 
the welfare of the students and, perhaps, the trainee's 
welfare as well. Often, even the accomplished researcher 
does not have the latitude to work out or develop high 
nsX strategies to completion. Using a simulation, how- 
ever, allows one to make mistakes in a safe environment. 

(2) The time period to run an educational experiment, complete 
with data collection and analysis, is frequently a year or 
more. This large time lag makes the real world an ineffi- 
cient environment for learning research principles. The 
most important advantage of simulation is its capability 
for rapid feedback . In a simulate^- school system, the 
resuTtscf an ''experiment" are available in a few minutes. 

(3) In field research, it is difficult to compare strategies 
empirically because even small changes in the research 
environment may spuriously affect the results. In a 
simulated environment, researchers can conduct either 
simultaneous or sequential experiments on the same po- 
pulation without confounding the experimental result?. 
Thus, "imulation enables us to assess the effectiveness 
of vai JUS research strategies by direct comparison of 
the results obtained. 



(4) Funding for competent re:.carch/evaluators is difficult 
to obtain. It is likely to be impossible for the be- 
ginner. In a simulation, we can practice handling 
budgets without risking the loss of real m, -y. 
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PLAYEKb' INTRODUCTION: Page 3 

OVERVIEW OF m GAMt 

In the FtHR-PRACTICUM game, you are a member of a toani which has been 
hired to "solve" a research/ evaluation problem which requires you to make a 
decision among several specified educational alternatives. The team collects 
information on which to base its decision by conducting a research project. 
However, in FEHR-PRACTICUM both the research environment (Fair City) and the 
behavior of the research subjects (i.e.. the students and teachers) are simu- 
lated by a computer program. Therefore, /our team cannot visit the research 
site in the usual majuier. Instead, you must collect all your information via 
a special information line which connects you to the simulated school system.^ 

FEHR-PRACTICUM is best viewed in two dimensions, the resource dimension 
and'' the process dimension, as illustrated in Figure 2. On the right the pro- 
cess by which a team "solves" an evaluation problem is defined. On the left 
are the physical components from which the players obtain the information 
required for their problem- solving activities. Typically, the solution process 
can be divided into two parts: the descriptive phase and the comparative phase. 

In the descriptive phase , each team is concerned with obtaining an ade- 
quate definition of the problem-its nature, severity, and extent-and in 
determining what other .people (i.e., past researchers) have done to remediate 
the problem. To accomplish this task, players must review the past research.- 
in the fie-ld (the Information Bank is a simulated library) , relate the research 
findings to their knowledge of the community in which the problem is set (Fair 
City), and conduct surveys (via the Data Generator) using appropriate tests 
to determine how many students are affected and how severe the problem is. 

During the comparative phase , each team is required to design and conduct 
an "experiment" to. compare the effectiveness of tha available treatments with 
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PLAYERS' n^TRODUCTIOM: Page 5 
students of various characteristics. The teams then analyio the results . and 
decide "^which treatment the schools should use with each type of student. Lach 
team's decision is submitted to the game manager who "opera tes^' the system with 
that decision in the computer simulator. The computer has the capacity to try 
one treatment with a student, then set him back where ho started and try another 
treatment. It is therefore possible to compute each team's "decision effective- 
ness index/* which is the ratio of t^e total growth Clea^ning) obtained under 
the team's decision to the total growth possible if each student wer.^ assigned 
to the treatment which maximized his growth. In addition, the computer prints, 
for each available treatment, a set of summai^ statistics which describe the 
characteristics of the students whose growch was maximized by that treatment. 
At the end of the game, the game manager, consultant Cs) , and players meet together 
in a consolidation session at which the decision results of each team are 
critically evaluated and the methodological implications discussed. 

Players can best comprehend the nature and scope of the research projects 
possible in the FUHk-PRACTICUM system by imagining that a real school system 
exists at the other end of the information line, and that the computer program 
is a '^research assistant" who will do exactly what they ask--no more, no less 
Although it is not able to converse with players, the program can perform the 
following tasks: 

(1) Search the school files and return information such as the grade 
and past or present achievement scoi*es for an individual student, 
or for all students in a particular school or class. 

(2) Administer tests, attitude scales, or questionnaires to individuals 
or to a group of students and return the resulting scores. However, 
in any one FHHR-PRACTICUM problem, the only tests which can be ad- 
ministered are those listed in the variable catalog which is pro- 
vided at the beginning of the game. 

(3) Find and print out the names (ID numbers J of subjects who have 
patterns of variable scores of a pre-specif ied type. For example, 
it could print out the ID's of all students in grade V who are 
maiy and had IQ scores less than 100- 
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(4) Administer any specified treatment (eJuLMticnal program) to stu- 
dents identified by individual ID'S or to groups of students 
identified by school, class or a pre-spocified pattern of variable 
scores. Since tests can be administered at any time, they caii be . 
used to determine the effects of a treatment over time. ■ 

In FEHR-PRACTICUM research, as 
in real life, the type of research 
design chosen is frequently dependent 
on the amount of money available for 
research. At various points in the 
game, your team must apply for re- 
search grants. Throughout the game, 
each team member is paid a salary. 
Each test administered and each 
. treatment applied also has a cost 
attached. These services are paid 
for out of a special FEHR-PRACTICUM 
checkbook, which is set up to help 
you keep track of the monies spent. 
Cin locations which have a com- 
puterized FEHR bookkeeping program, 
the checkbook will not be used.) Thus, one of a team's major tasks is to plan 
its research so as to ensure that it obtains sufficient information to permit a 
valid decision without exceeding its grant funds. 

FEHR-PRACTICUM is a game in that several teams normally attack the same , 
problem and compete for the "best" solution. However, the competition is parallel 
rather than direct, since the actions a team takes cannot affect another team's 
solution in any way. It should be pointed out that there is no "right" experi- 
■ent to perform and no predetermined "correct" decision. In addition, a team need 
not decide to use the :;ame treatment for all subjects; it is entirely reasonable 
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to assess the rulati\e merits of the teams' research procedures by compaimc 
the sets of variable scores which are obtained after "operating" their 
decisions in the simulated system. This capacity for feedback on the quality 
of a researcher's work is considered the most valuable aspect of the FEHR- 
PRACTICUM model. 

Operating Staff 

The FEHR-PRACTICUM game requires 
the services of a game manager 
and at least one research con- 
sultant. The function of 
each of these staff mem- ^ 
bers is explained 
below: 

The Game Manager 
is a senior research 
specialist who has been trained 
to supervise the game. His major 

task is to act as a Uaison between the players Cor teams) and the simulated 
school system. All requests for information and all replies to these requests 
must pass through his hands. 

The research consultants in the FEHR-PRACTICUM game serve the same function 
that they would in real-life research. V^lienever a player (team) is uncertain 
about any aspect qf tU^r research methodology, they may hire a consultant to 
help them. If a number of "teams are competing .n a particular game, several 
research consultants may be provided. The game manager will announce at the 
beginning of each game the number of consultants available. In addition, he 
will provide the teams with a vita on each consultant to help them decide which 
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perscm to hire fox «,y one task. You may hire^a consultant at any point during 
the ga«e. providing one is available-at any point in tin. it is possible for 
all the consultants to be engaged by other tetts. The cost of consultant 
service will vary according to the qualifications of the person concerned. In 
general, consultants charge for half-hour unit, at a rate approximately equal 
to their per diem rate in real life. 
Physical Components of the Game 

Physically, the FEHR-PRACTICUM game consists of four interacting components 
an information bank, a data generator, a message generator, and an in-service 
training CIST) unit. The function of each of these components is described below 

CD Information Bank . The Information Bank is actually a cross-referenced 
file. Historical information about the Fair City system, statistical data 
about the tests used in the problems and abstracts of the real-*iorld studies 
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CHECK OUT 




referred "to in the problem booklets are bolmd together by problem (R.F.P.). The 
Bank may be used as the only reference source in a fast version of the game 
(but in a slower version. ^here textual information is important, it should be 
c^nsidel-ed a mere collection of abstracts), or it. may be augmented by further 
research into the original sources and/or other materials. The choice depends 
on relevance of the problem content to the educational goals of the players in- 
volved Information Banks may be obtained from the game mana^ger at any point in 

the gfune. • 

(2) Data Generator . The data generator is a computer program which simu- 
lates the behavior of the individual subjects within our educational system.' 
Each sub^Hc{ is described by a unique set qf scores for a large number of 
variables such as sex. age. number of siblings, sibling position, intelligence 
ai,d various attitude scales and achievement tests. However, an individual's 
scores for many tests-especially attitudes and achievement -will change over 
time. TTie direction and rate of these changes represent the "behavior" of the 
individual. 

Within the data generator, three types of information gathering processes 
may be used: 

24 

-2b- 



PUVERS' INTRODUCTION: Pag^ 10 

- (a) File Search . A file search will retrieve the inioxmation 
which exists in the filers of the school system. This may 
be considered fixed information in that you will get pre- 
cisely the saae scores for each individual each ti»e you 
search. 

(b) Survey . Data which can be obtained by administering a 
test at the present time is available through a survey. 
Sinct ■•asureaent error is involved, a survey will return 
a slightly dijfferent test score for each student each ti-ne 
it* is uied. 

(c) Treatnent . The treatment process enables tie player to 
adBinister any available test to subjects at various points 
in ti»e. Since players may also control which treatitents 
are given to, any individual (or group) , thils process enaljles 
you to assess the effects of various treatments experimentally.. 

(3) Messa£e Interrupts . It frequently happens that a research project 

is radically changed by external eyents which the 
experimenter cannot anticipate or control. For 
example, a teacher s::rikB which interrupts an experi- 
ment may dunge ^vpil attitudas as wel^l as introducing 
costly delays. Such "acts of God" may be introduced 
into the FEHR-PRACTICUM game by message interrupts. 
At sotte time(s) during the game you may be given a- . 
message by the game ma]iagej< Somrvof these will be 
relatively ui«important such as nptification oi the 
status of the computer system. Others, however, may 
require you to make adjustments in your research plan. 
For example, a message that your research budget has 
been cut might ^necessitate the use of smaller samples. 
Such messages are intended to provide experience in 
dealing vith the unexpected. Message interrupts are an optional feature, to 
be used at the discretion of the game manager. 
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(4) IST.Units. in real life, a Tosearcher muy need extra schooling to 



enhance his knowledge in areas «levant to hir research. Therefore, he might • 
opt to take- course^ a nearby university. Tn the FEHR-PRACTICUM game, such 
■"ccJujses" are called I^nits . The in-service training unit consists of a 
set of instructional matr^s which provide tr'^ning in such areas as writing ^ 
format statements, criterion referenced measurement in research and the like. 
The "courses" which (are availaUe are listed in the Players' Instructions. ' 
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FEHR-PRACTICUM research takes place in Fair City, U.S.A. Since the con- 

-text in* which a p rob lo- ^ 
exists frequently has sig- 
nificant i»pUcations for 
:Lts solution, it is impor- 
tant that you read the Faxr 
City booklet before coming 
to the next s#ssior. P4ease 
pay particular attention to 
the sectibns describing 
education. 

Since you will use the 
infonatlon provioed in this 
booklet in the practice 
game and in any subsequent^ 
gimes you play, please have 
it on hand at each game 
session. 




GOOD LUCK I MAY ALL YOUR PROBLBIS BE FEHR ONES! 
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PLAYER'S INTRODUCTION TO FEHR-PRACTICUM 



This addendum appears only in the Game Manager's Manual. It pro- 
vides an overview of FEHR-PRACIICUM materials^ ihd describes the function 
of each component and the interrelationships among them. 



MATERIALS . 

The FEHR-PRACTICUM materials can be classified on two broad dimen- 
sions. The first of these is the access dimension. Where is the material 
physically located? How and by whom is it normally accessed? The second 
dimension concerns the generality of the materials, whether it can be U5a<J 
in all problems (content areas) or not. ) 

The access dimension is sub-divided into four categories. Category Y 
contains all the Data Generator materials. These would normally be accessed 
at the local computer center or a remote terminal. Category 2 contains 
materials which would normally be used only by the Game Manager and/or 
those planning the instructional uses of the practicum. Category 3 con- 
tains materials which are shared among players. -These would normally be 
accessed in a laboratory-classroom. Category 4 would contain all the mater- 
ials normally provided to each player-trainee. 

The generality dimension contains two categories. Category 1 (common) 
contains all materials which can be used with all eight problems, while 
categcy c (unique) contains materials which change from problem to problem. 

The entire set of FEHR-PRACTICUM materials, categorized by access 
and generality^, appears in Table 1.' In the discussion below, there is 
additional descriptive information for every component except the Game 
Manager's Manual which has been described in context. 

Cc Mnmon Materials 

The Main Data Generator (main computer program) consists of a -c of 
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TABLE^ 1 : FEHR-PRACTICUM MATERIALS CATEGORIZED BY 
ACCESS AND GENERALITY 





ACCESS 




GENERALITY 


DIMENSION 


DIMENSION 


1 


COMMON TO ALL PROBLEMS * 


2. 


UNIQUE CONTENTS FOR EAC^ PROBLEM 


1 

1 • 


rOMPIlTFR 


a. 


Main Data Generator. 


a. 


Data Generator Problem Packet: 








(Main computer pro- 




unique program parameters for 








gram. ) 




each problem (Eight separate^ 












packets. ) 


2. 


GAME 


a. 


Game Manager's Manual 


a. 


Hamp Mananer's Manual: 




MANAGER 




Sections I & II 




Cprtion*; ITT & IV 

wCUCiUild ill U 1* 


3. 


LABORATORY 


a • 


FEHR-PRACTICUM 1ST 


1 

a. 


Information Bank: mater'^al sep- 




OR CLASS- 




Units. (Five sepa- 


■ 


arately bound for each problem. 




ROOM 




rately-bound units.) 




(Eight separate Information 






1 

1 


Note: Some Game 


! 

\ 


Banks. } 






t 
1 


Managers may wish to 




* 








supply a copy of this 












material to each 


1 










player. 


1 
! 


; 






I h 


Po^ov^onr'oc * ^nnnl ^ pH 
ncTcrcriLcb * ^vUppI 1 cu 


! 










locally. ) 






4. 


PLAYER 


a. 


Player's Introduction 




RFP (Request For proposals) 




MATERIALS 




to the/ FEHR-PRACTICUM 


! 


Document: a soecific description 












of the particular problem to be 






b. 


Fair City, U.S.A. 




investigated, separately bound 






c. 


Player's Instrucions 




for each problem. (Eight sepa- 








for FEHR-PRACTICUM. 




rate RFP documents. ) 
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punch cards (or a computer tape) containing the rORTRAN IV source prograiiu 
However, the main program cannot produce simulated data unless it is com- 
bined with one of the data generator problem pockets (see below). 

The FEHR-PRACTICUM 1ST (l^n Service Training) Units consist of separate- 
ly-bound, semi -programmed materials which provide detailed instructions for 
accomplishing specific tasks encountered in the practicum. The five units 
available are: 

I. Assessing Success For Complex Objectives. 

II. Criteria-Referenced or Mastery Testing. 

III. Computer Format Statements for FEHR Data. 

IV. Sampling The Subjects To Be Studied. 

V. Using The FEHR Secretary. 
The references to be supplied locally consist of any research-oriented 

materials which will assist the player-trainees. A list of suggested titles 
appears in a subsequent section. 

There are three p layer* s materials which are common to all problems. 
All three of these materials appear in this manual. The first, Player's 
introduction To The FEHR-PRACTICUM Game, was contained in the first twelve 
pages of Section II, which you have just read. The second and third booklets, 
Fair C i ty , USA , and Player^s Instructions For FEHR-PRACTICUM constitute the 
second and third chapters of Section II. Wherever necessary, the original 
materials printed herein have been supplemented by notes and addenda addressed 
to the Game Manager. 

Uniq ue Materials 

The total FEHR-PRACTICUM system provides a choice among eight major 
problem^♦dealing w h eight different content areas and involving students 
at different educational levels. Each problem has its own unique Data 
Generator Problem Packet, Information Bank, and RFP (Request £or Proposals) ^ 
document. Although the specific contents differ, the format of each of 
these components is the same for all problems. ^ 

The Data Generator Program Packet combines with the Main Data Generator 
(above) to form the complete computer program' necessary to operate a specific 
prjolem. When the fjirst order is received from a new user, an intact deck 
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contaimng both the Main Data Generator and the Data Generator Packet 
for the standard REMAR (Remedial Arithmetic) proWem is shipped. When you 
are familiar witti this problem new ones may be ordered one at a time. 
Since the maia sections of the program are cormon across problems, we routine- 
ly ship only the Data Generator Problem Packets to users who already have 
one or more FEHR problems operational. 

The Information Bank contains a brief summary for a number of real- 
life articles related to the Content area of the problem, plus descriptive 
information for each of the standaradized tests available in the problem. 
These are printed on loose-leaf sheets, one article per page, and arranged 
alphabetically by author. This format was adopted to permit users tc up- 
date the information as new relevant research becomes available. 

The RFP document for each problem has the same general fortn. Page 1 
identifies the content area and the educational agency which is sponsoring 
the research. Page 2 provides a general narrative description of the problem 
and refers the reader to an appendix from which more specific details may 
be obtained. Pages 3 to 6 contain the checklist of Tasks to be Performed: 
a detailed listing of all the tasks involved in a complete practicum. The 
Game Manager chooses those tasks best suited to the local needs. 

The detailed content of each RFP is contained in a set of appendices. 
These are usually five or six appendices containing the informatiort described 
belowt 

Appendix I - Information Bank Material . This Is a list (or index) of 
all the abstracted articles which come along with the simulation. 
Going through these articles will give the player an overview of the 
research in the area. If a player is especially interested in the 
substantive area to be investigated, we suggest that the Information 
Bank be used to determine which articles should be read in full. In 
addition, the Information Bank provides nomative data (means, stan- 
- dard deviations, reliabilities, etc.) and a description of the test 
content for all standardized tests listed fo- that problem. In addi- 
tion, for many tests a critique is also available in the Information 
Bank. 

A ppendix II - Research Population . Appendix II contains a complete 
list of all subjects available in the problem and explains how to 
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interpret the student ID. 
* Appendix III * Catalog of Treatments . Appendix III is a list of the 

treatments which can be admini^stered in the problem, a list of costs 
of each treatment, and a definition of the time sequences used in 
the problem. 

Appendix IV - Catalog of Variables , Appendix IV lists all the vari- 
ables available in the problem, the costs of «ach variable score, and 
the conditions ynder which the vaViable scores (test scores) may (and 
may not) be obtained. 

Appendix V - C omnittee Report , In nearly all problems there is some 
preliminary information^dvai lable such as why particular treatments 
were chosen an(^ what previous research the school system has done 
In the drea, A concise^ summary iof this information appears in Appendix 
VI. 

OVERVIEW OF AVAILABLE PROBLEMS 

A total of eight problems are available in the FEHR-PRACTICUM system. 
Once the Game Manager and research consultants have become thoroughly fam- 
iliar with the FEHR-PRACTICUM system, it is possible to conduct a practlcum 
in which several problems (indeed, all eight) are being operated simultaneous- 
ly. This has the advantage of permitting player-trainees to choose the 
area closest to their own substantive interests. Although the problem-solving 
procedures are similar from problem to problem^ we have found that the choice 
of content area can make a tremendous difference to a trainee's motivation. 
However, managing FEHR-PRACTICUM is a complex task. We strongly recommend 
that new users not operate multiple problems until they have had at least 
two or three sessions practice with a single problem. 

During our field validation studies, we tried training Game Manager's 
^ on several different problems. It was o>t experience that prospective Game 
Managers who were supervising problems based in a content area with which 
they were unfamiliar had great difficulty concentrating on the management^ 
tasks per se , and consequently learned much more slowly than those supervis- 
ing problems with which they were familiar. Since it was also found that 
•numerous demonstration file searches, surveys, and field experiments facilitated 
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learning, it was decided to develop training materials based on one starriard 
problem and featuring man> practical examples. The REMAR problem was chosen , ^ 
for this purpose because remedial arithmetic is the one content area with 
which most educato*^ have had some experience. Section III of this manual 
contains prograumed directions and a number of practical examples for each 
task in the REMAR problem. After completing this section, most people will 
have no difficulty administering a full-fledged practicum session using 
\ the REMAR problem, the extension to other problems i" then just a matter 
of becoming familiar with the content area by goingf through a complete prac- 
ticum following the Player's Instructions step by step. 

The eight available problems are described below, beginning with the 
standard REMAR problem. For your convenience, the instructional strength 
and limitations of each problem are listed under the heading Special 
Characteristics . For sessions in which the specific content of a problem 
is of substantive' interest, we strongly recommend that copies of the primary, 
references listed with the problem be made available. 

Project REMAR Remedial Arithmetic. (RFP002). This is the standard problem 
used for the first implementation at a new site. The RFP is issued by the | 
Fair Cit' School Board, who are concerned at the growing number of grade 
seven students who cannot do arithmetic computation well enough to succeed 
■ in the regular grade seven curriculum. You are asked to conduct field 
tests to evaluate the effectiveness of three proposed new remedial arithmetic 
programs as compared to the current practice. At the conclusion of the 
project, each team must decide on the basis of their experiment which of 
the new program (if any) are to be implemented. (Note: it is entirely 
possible to reconmend different programs for students of differing character- 
istics.) 

Particulars : Variables = 78, Treatments » 4, Total Research Popula- 
tion » 1906 seventh grade students fro^ seven junior highs, each 
with several classes. 

Special Characteristics : This problem permits direct experimental 
comparisons. It is fairly heavily orientated toward criterion- 
referenced tests or sequential mastery teSts. Sampling, selection of 

4 
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variables, and design all are involved and can be accomplished fairly 
easily. Because of the necessity to select only the poor students, 
this problem provides a rich opportunity to study the effects of 
statistical regression. In addition, there are some conceptual dif- 
ficulties with respect to the precise definition of success in terms 
of variable scores, and soi:.e sticky statistical questions centering 
around the analysis of mastery-test c(ata. Nevertheless, this is 
perhaps the easiest, most straightforward problem, since the objectives' 
are fairly clearly defined in unambiguous terms. 

A unique feature of this problem the emphasis on the cost- 
effecti'Veness aspect of the decision among programs which is intro- 
duced by a wide disparity in program costs and a posit,ive correlation 
between cost and learning. 
Primary References (None) 

Project PEP : Perceptual Education Problem, RFPOOl . The term "perceptually 
handicapped" has been used in recent times to.identify a large number of 
children who have normal intelligence but because of a "perceptual problem" 
have great trouble in school, particularly in reading and writing. In this 
problem the players are requested to aid the Beard of Education and a com- 
mittee of teachers in deciding such questions as: Does Fair City need a 
Perceptual Education Program: Are psychological and socio-economic variables 
relevant? Which treatment should be recommended? 

Particulars : This problem has 161 variables, three treatments (2 ex- 
perimental, 1 control), and a total research population of 426 
students in four grades of one school. 

Special Characteristics : Thii problem permits a direct experimental 
comparison of the two proposed programs with present practice (the 
control). It features the usual sampling, variables selection, and 
design difficulties, with emphasis on the last two. One of the major 
difficulties in PEP is problem definition. The School Board sees 
the problem as a lack of achievement. How does that relate to "per- 
ceptual handicaps'? What is a perceptually handi-capped child? 
What variable scores signify a handicap? The conflict between end- 
result variables (achievement) and intermediate results (changes in 
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variables which are hypothesized a$ pre-reqiiisite to successful 
achievement) raake this problem particularly useful for practicing 
problem conceptualization skills. An additional feature is the 
"bOllt-in" experience with regression toward the mean which is caused 
by stringent selection criteria. 
Primary References : 

Frostig, Marianne. "Visual Perception in Brain-Injured Children." 
American Journal of Orthopsychiatry, 1963. 33 (4), 665-671. 

Johnson, J. & MyklebuSt, H. R. Learning Disabilities : Educational 
J>nnciples and Practices , Grune & Stratton, Wew York, 1967. 

Kephart, Newell C. (U. of Purdue, Lafayette) "Perceptual -Motor 
Aspects of Learning Disabilities." Exceptional Children , 1964, 
31 (4), 201-206. 

McCarthy, J. .1. & McCarthy, J. F. Learning Disabilities , Allyn & Bacon 
Boston, 1969. 

Project HEADSTART : Early Childhood Education. (RFP004). The Fair City 
School District believes that there are a growing number of children who 
are entering first grade ill equipp«d to perfom at normal levels. In 
this problem the players must aid the Board of Education in deciding if a 
Headstart program should be introduced and which particular program best 
meets the ntads of the Fair City children who require extra attention. 
The players should not only decide if there is a need but also if the 
gains made in Headstart are retained after the child has entered the regular 
public schools. 

Particulars: Variables » 78, Treatments » 7, Research Population = 
1822 or all three-year-olds in the city. 

Special Characteristics : Although direct empirical comparisons among 
the 7 treatments are possible, this is too complicated to permit in 
practice. An additional complication is the fact that not many 
measures are available for pre-schogl kids, let alone many reliable 
ones, particularly those which miy change as a result of some program. 
In addition, the problem requires long-term (longitudinal) assess- 
ment of changes, and most available tests — even though they have 
the same name — have different norms for different age groups. 
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Primary Reforences : 

Weikert, D, "The Perry Preschool Project in Ypsilanti, Michigan," 
1969. OE- 37035. 

Project READ : Reading Assessment Problem (RFP005). The teachers of the 
primary grades have become dissatisfied with their present reading program 
because of the increasing number of students who are falling behind ttieir 
.peers in the development of reading skills. In this problem the players 
are to aid the teachers and principals of the elementary schools to detennine 
if a new reading program should be instituted in Fair City. One. of the 
questions they will answer in this, problem is whether there is one curriculum 
which can best meet the needs of all Fair City children. 

Particulars : Variable's = 170, Treatments = 3 (2 experimental, 1 
control) ,• Research Population = 2000. 

Special Characteristics : This problem permits direct experimental 
comparison. The major emphasis here is assessment in terms of mul- 
tiple behavioral objectives. About naif the variables in this proi)lem 
are criterion-referenced. Consequently, variable selection is an 
i.nportant element in this problem. But perhaps the major feature is 
the data interpretation task. The multiple successes and failures 
of studeits in various programs must somehow be summarized in a 
conceptually meaningful way to permit program-to-program comparisons, 
and a subsequent decision among programs. . 
Primary References : 

(Not used in development, but very similar and helpful) 

Duffy, G. G. & Sherman, G. B., Systematic Reading Instruction , 
Harper & Row, 1972. 

Proje^TQUEST : Validation of a Teacher Questionnaire {RFP006). The pur- 
pose of this project is to validate a q\jestionnai're which "evaluates" teacner 
performance at the college level. The questionnaires are to be administered 
to students presently enrolled in college classes. The players are to assist 
the administration in this project by comparing the effects of feeding back 
informction from various sub-scales on the new questionnaire. A second 
questionnaire and several achievement scores are also available to be used 
in the validation task, 
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Particulars : Variables = 60, Treatments = 4 (3 types of feedback, 
1 no feedback). Research Population « 512 university students from 
20 different classes. 

Special Characteristics : This problem contains two questionnaires. 
(1) the old questionnaire, and (2) a new one designed to provide 
more information. It opens up quite a can of worms — how does one 
validate such a thing as a student evaluatiory pf teacjiers? In this 
problem, ,the path suggested is to see how effective the questionnaire 
tn^hangiBg_4mif essors.' 4 , as^tneasoretf l5y the ques^TohnaTre . 

The variables (questionnaire items) are all 5 option attitude items. 
Individual item responses may then be combined to form various scales 
which relate to the developers' (i.e., the University Committees) 
concept of teaching effectiveness. Feeding back to. a professor his 
"scores" on one or more of these scales from last semester should 
influence his score (on the scale(s) concerned) this semester. Thu^ 
it is possible to collect evidence of the construct validity of the 
questionnaire. 

It is important to note that, while we give Question mi res to stu- 
dents, the unit of observation is a teacher (i.e., the class). This 
introduces a variety of 'Interesting statistical questions which are 
an important aspect of this problem. 
Primary References : 

Cronback, L. J7 Essentials Psychological Testing . Chapter 5, 
"Test Validation", Third Edition, Harper & Kow, 1970. 

Project RMA : Remedial Math for Adults (RFP007). The open enrollemnt pol- 
icy and the wide variety of people who attend community colleges necessitates 
the provision of additional support services for students and citizens. 
In this problem the players are requested to evaluate the remedial math 
course at the Fair City Community College. This course Is intended to pro- 
vide the students taking it with the skills to do college level work. The 
players will be asked to answer such questions as: Does a remedial program 
work for adults? For what kind of person is this progran. least useful? 
How can the program be made more effective and efficient? Should the course 
be continued? 
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Pa rticulars : Variables = 26, Treatments = 1 , Research Population 
= 251. 

Special Characteristics : This problem is strictly a post-hoc evaluation 
task, with evaluation made solely on the basis of Evidence collected 
during the semester. The trickiest part of this problem is. an 

\ operational definition of success. Since the students^me from a 

HidF-vartety^oT bacRgroulidsT and T)^ave very different goals, the 

, meaning of success- varies from group to group. The selection of 
relevant variables can also get intensely complex, since there are 
several varieties of attitude, aptitude, and achievement measures 
which the teams might be used in any combination. 
Primary References : „ . 

Dalke, Richard M. A Case Study of an Individualized Course in Arith- 
. metic at a Corrmunity College. Dissertation, University of 
Michigan, 1971 f ^ 

Project BUS : Busing to Achieve Integration (RFP008). In response to re- 
cent Supreme Court rulings, the Fair City Board of Education has decided" ^ 
to integrate the city's schools through busing. As in most cities, the 
people of Fair City have very strong views about busing and there are 
many complications to consider. In this problem the players are requested 
to evaluate the effects of busing in the city's, elementary schools to deter- 
mine its advantages and disadvantages and what are the sources of the 

problems that exist. 

• Particulars . Variables = 52. Treatments = 1, Research Population = 
3.633 pupils in grades 1 and 4 of 21 different schools. 
Special Characteristics: This problem is strictly a post-hoc evalua- 
tion. Teams are call i in after the busing decision is made. Al- 
though on J the first and fourth graders are available to the players, 
there is nevertheless a large number of subjects from a v,ide variety 
of sot^o-ethnic neighborhoods. Since computer costs prohibit an ^ 
exhaSive survey, this problem offers a rich environh»^t for practic- 
ing sampling skills. There is the fact^that any resul^JPay-be 
attributable' t;^. the new organization in' the eleme-tary schools rather 
than the busing plan itself. There is also extensive direct measure- 
ment e^^erience. since it may place more weight on attitude measures^ 
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as criteria. The relatively low reliability and validity of such 
measures and the categorical nature wm necessitate the construction 
of broader construct variables through various combinations of ques- 
tionnaire responses. 
Primary References : 

Sullivan, Neil U. and Steward, Evelyn A. "Now is the Time: Integra- 

t+oft in the Berkeley-Schools," Indiana Universij:y Press, 

BloQiriington, 1969 (ERIC). j • 

Project EXTSY : Extended School Year (RFP003). What 'are the benefits t," an 
Extended School Year? Fair City, like many other areas, believes there would 
be economic'if not educational benefits, from having the school schedule re- 
organized so that the schools are in operation all year arouno. The players 
ere to investigate the situation and determine which, if any, schedule has 
the most advantages for Fatr City. 

Particulars : Variables = 56, Treatments = 3 (2 experimental, 1 con- 
trol). Research Population =12,393 students from 21 elementary 
schools. ' 

Special Characteristics : This problem .permits some ^experlinental com- 
I pari son, but notice that an entire school must t^e assigned to any one 

/ ' treatment. This introduces some interesting questions with respect 
to the appropriate unit of obseirvation and the general izability of 
results. EXTSY involves extensive sampling, variable selection, and 
design problems. One of the difficulties in this problem H the fact 
that since treatments must be administered to intact school s\ no true 
experimental design is possible. Attitude variables may be more val- 
' uable here, but reliability and- validity problems, as well as the nom- 
inal nature of such data, are problematic. The fundamental question 
is "which is more important: achievement, cost, or popularity?" Sev- 
eral unique aspects of this problem are introduced by its longitudinal 
nature (3 years). 
Primary References. : 

Department "of Education, New York. "The Impact of a Rescheduled 

School Year." A special report prepared for the Governor and the 
Legislature of the State of New York. The University of the State 
of New York, the State Education Department, Albany, New York, 
March, 1970, 158 pages. 

* • ' 
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The 

Friendly 
City 



This book is a gift to you from the Chamber of Co.i»nerce of Fair 
City USA. If you now live. work, or transact business^n our 
city', or intend to do so in the future, this book should prove 
both useful and int -esting to you. It contains information 
abovt our people, quarters, government, municipal services, 
commerce and industry, health and welfare, theater and the 
educational system. ^ 

Friendship is the foundation of Fair City's vitality and strength. 
Wherever this spirit appears in the pages to follow, something of 
benefit to the whole community has been accomplished. Whenever 1 
the spirit of friendship appears in the future, we will accomplish 
„ore. in Fair City, "the friendly city", we have struggled ^ard 
by friendly co-operative effort, and if we are to move forw^d. xt 
will be by the same means. 



CC 



Fair City 
C'^amber of 
Commerce 
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FACTS & FIGURES 



POPULATION: Fair, Ci^y - 122,594 i 


. c . ' 1 

irtOADCAST FACILITIIS: Four radio 


AREA: City - 31 square miles 


stations, two TV stations and cable . 

^ i 

SIEWSPAPERS: Star-Gazette (daily 


TEMPERATURE MEANS: 


jrli.ll llll villi mj / , ^ 

Vlaximum - 60,8*" 
Mean - ^o* i 

PRECIPITATION: 


circulation of 50,000), Burns County 
Reporter (weekly distribution of 1400) 

RAILROADS: Erie-Lackawanna, Central 


I 

BUS L-INES: Four intercity and one | 


Kain jj. / incnc5 jJci /cai 

Snow - 40.6 inches per year 

GOLF COURSES: Three 18 hole courses 


county line \ 

i 

TRUCK LINES: 35 carriers to all , 


principal cities \ 
AIRPORT: Burns County Airport 


WORJC FORCE: 53,849, ir.cluding 17,000 


in manufacturing induitries 
PAY RATES: April 1970 average 


SHOPPING AREAS: 5 prin5:ipal ones i 


— 1 

THNDTCT QTrHTQ AMD HISTORIC ATTRACTIONS: 


manufacturing pay - $3.03 per hour 
($119.99/weeV^ 

MANUFACTURING INDUSTRIES: 112 


Early American Brewery 

Harding,!^s Summer Home \ 
Fegans Art Gallery 

Cervenka Gallery - Utopia State ; 
Burns County Historical Center ^ 
Mohawk Battlefield Reservation 

TOTAL PROPERTY TAX RATE (1970): * 


RETAIL AND WHOLESALE OUTLETS: 74 7 


SCHOOL ENROLLMENT: 23,883 (IdSt Fall) 


PER CAPITA INCOME: $2,348 (1970) 


(per $10,000 unit ot tull Value) 
Ranges ^rom i205.15 in less populated 
areas to $467,95 for full service areas 

CHL^PCHES: Over 100 churches represent ingj 
24 denominations, all major faiths ; 

SCHOOLS: ! 


■ MEDIAN FAMILY INCOME: $6,943 (1970) 
(national average: $4,630) 

RLFAIL SALES: $192,967,000 for Burns 


County, including $150,185,000 for 
Fair City (1970) 

NEW rlOUSlNO VALUt. >D,oifD,yH^ 


Public - 21 elementary schools 


BUILDING PERMITS VALUE: $19,957,709 


7 junior high schools 
3 senior high schools 

Parochial - S elementary schools 
1 senior high school 


(1970) 

HOSPITALS: 1- Tc^al bed capacity 545 


HOTEL AND MOTEL ROOMb: 600 


(air conditioned) 
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Fair Cit> , U.S.A.. is a modern 
N$jnerican city of today--and r nuniovsl 
[with its combination of the beM as- 
pects of city living and. ready access 
to the beautiful Burns ^ounty countryside. Fair City is the place for YOU to 
live! 

The city itself is crossed by two rivers: the North River flows tt^rougti 
the heart of the industrial district and the 
South River lends graceful charm, to the muni- 
cipal area as it w .»ds its way through the 
historic site of the city.. 

Both rivers flow toward the city from the 
surrounding countryside. 1*he N-^rth River flows 
through the scenic valley northwest of the 
city; the parks and picnic sites in this area 
attract many of the citisenfi of Pair City diuf-| 

ling their leisure 
[hours and vacation days. 

The South River runs through the rural area 
[southwest of the city past the many picturesque 
farms that dot the landscape. 

East of the city, the rivers converge near 
^^Ithe new shopping, mall that has provided the Fair 
III > > MW l MCity residents with year-round, climate-controlled 
I^^^^^J^IH^^^^I stuping » banking, eating and entertainment. 





INDUSTRIAL AREA 



MUNIC7PAL AREA 



- 49 - 

45 



FAIR CITY: Page 6 




FAIR Cm:1»age 7 



Qovenmieiit 



in 1908, 
mayor and 



Fair City's municipal govern 
ment is a general- purpose govcnujicnt 
operating under a charter adoptoj 
The governing officials are the 
a $everi-member CxV Council; tlte^ 
are elected at- large in November of even- 
numbered years. 

The city has been alternately governed 
by Republicans and Democrats for the last 
twen^^y years. The first black to b^ elected to 
the City Council was John Newton, in the last 
election. 

The inciimbont mayor, Frank Friendly, is 
a Democrat. He prides himself on the pity's 
progressive attitude toward contemporary urban problems and the wtent of 
citizen participation in community affairs. His current term of office expires 
in two years; 

Membership on the present City Council is as follows: 




Lewis Left 
A.M. Biguous 
Stanley Staunch 
A.M. Lucre 
John Newton 
Harvey HalfWay 
Reginald Right 



PARTY 



Draocratic 
Democratic 
Republican 
Democratic 
Democratic 
Republican 
Republican 



STATUS 



Appointed, 1960; Reelected thr%e times 

Elected last year 

Appointed. 1968; Reelected twice 

Elected, 1966; Reelected once 

Elected iMt election 

Appointed, 1966; Reelected twice 

Elected, 1966; Reelected once 



The Fair City 
Council meets at least 
weekly and often more 
frequent ly . Disagree- 
ment and debate are 
common in Council meet- 
ings. Citizen groups 
are often heard. Council 
members are not reluc- 
tant to take public 
positions on issues 
and air them at Council 
meetings or in the 
press . 
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THE QUARTERS 




The residents of Fair City represent a wide variety of ethnic groups. 
Currently, the popul^.tion is 62% white an 38% non-white; "°"-';'55^®-..ui- ,1) 
populatioA has been increasing slowly over the past 20 years (Appendix- .able li; 

Fair Citv. like all large cities, has citizens belonging to many 
socio-economic iove is and this fact is reflected in housing. The ^omes xn 
the city range from the modern high-rise apartment complexes near the new 
Shopping mall, to the lovely old homes found in the center of/^^e "^y* ^° 
the more modest middle- income dwellings in most of the city, to the large 
but dilapidated homeb on the south side of the city. 

Recently, some attempts at urban renewal have been made in this last 
area; notable among them hs been the construction of several hundred units 
-of low-income housing near the South River. 





MODERN HIGH-RISE APARTMENt COMPLEXES 
NEAR THE NEW SHOPPING MALL 
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Franklin Park , a quiet neighborhood near the center of town, consists 
of charming old hoaes on spacious lots. The residents of this predominantly 
white, iq>per-«iddle class area, tend to be engaged in professional, managerial 
and technical occupations. 

Fair Valley , although close to the industrial area, is a, pleasant section, 
with jmall homes and apartments. The majority of residents are white, of lower- 
middle class status, and are generally employed as clerical or skilled workers. 

Bakersfield , named for one of Fair City's leading industrialists, was 
originally developed by Mr. Baker's company for his factory "employees . The 
modest but neat houses are still inhabited by factory employees, both skilled 
and semi-skilled. This lower-middle, to upper-lower class section is fully 
integrated. 

Shady Terrace and North End both consist of very small, inexpensive houses 
and apartments . SITady Terrace was a low-cost housing development, while North 
End evolved independently. They are both resided in by black and other minority 
persons, of upper-lower class status, who are generally employed as s«ai-skilled 
or unskilled workers. 

Washington Heights , the most exclusive section of town, is a predominantly 
white, upper-class neighborhood. It consists of large, custom-designed houses 
on secluded streets and modem high-rise apartments. Most of the residents 
•re engaged in professional and executive managerial occupations. 

Iro quois Village is much like Franklin Park, with its lovely old homes 
and beautiful landscaping . The middle and upper-middle class, predominantly 
white residents, tend to serve as managerial, technical and professional people. 

River View , a once-wealthy, :>ld neighborhood now in decay, is inhabited 
by black and other minority persons of lower-class status. These people are 
employed as unskilled workers, although a large number are unemployed and 
receiving welfare. Some of the welfare recipients occupy the urban-renewal 
housing projects which were recently built in the area. 

The Do wntown area is the center of town, with both business and residential 
areas. This integrated area is the home of lower-middle class people, generally 
employed as skilled or semi-skilled workers. 

Maple Ridge , is an integrated neighborhood, including apartment buildings, 
duplex homes, and single- family dwellings. Its middle-class residents are 
generally engaged in managerial, sales and service occupations. 
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THE PEOPLE 

More than 50,000 residents of Fair City are employed in a variety of 
occupaUons' hanging fronv the products produced by factor.es a on the 
North River to the research enterprises associated with Utopia State uni 
iersity "urrently. .ore than 93 percent of the ^otal wo.k ,orce is emp- 
loyed giving the city an average household income ot $6,943. . Be^ow ^ 
map i;dicating the location of the various neighborhoods comprising Fair City. 



Shady 
Terrace 



North End 
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Education 

COKMUNITY 
SCHOOL- OBJECTIVES 
AND PROGRAMS 




The Fair City school system is typical of a system which has grown steadily 
in.iccent years. It has many of the characteristics of a s"!*^ . ^^^f f;^,^J 
the community and parents are vitally concerned ana interested m the operation 
of the schools, and teachers are dedicated and honored ciUzens of J^e town^ Yet 
with the gradual increase in the population, especially iif*the black population, 
the school system has had to reorient itSelf. 



_ „ In 1970 the school system population contained 23.885 
pupil s o7^which"36\"wer e black (Appendix-Tables 4 and 5) The lower class areas 



\ 



SaWOL POPULATION: 



PUDllS or wnicn 00» were uinf^r. v^l't"^"-*'' ^ 7' , ^««<.o<r«rl 

?wtained predominantly black pupils while the upper middle class 
few. if any. black pupils. This can be seen by comparing the map on the opposite. 
IZe with i^ie map oS Jage 8. Recently, this trend toward ^jP^-^^f 
has been quickly reversed in the lower middle class areas, but some work i- 
needed tc a.'-viate the possibility of problems in ether parts of 
pLms have b..n considered by Fair City to help solve this racial imbalance but 
^ne has not been proposed which satisfies the various! factions withm the city. 

'\ ADM lMgTRATION : The new superintendent. Mr. A. Luirnus was a J^^duate of 
thes chool system he re. He attended the largest and oldest ^l^J^^i^jy ' I 

Jack^.djill Elementary, as well as Gordon Jm.ior High »^«^Jf J^^i 

Mr. LuJius v.as brought in from an urban school system so that he ^2"ittv cLsioers 
situaUons such as racial imbalance in the Fair City schools, ^a^' ^ity consiaers 
L self fortunate to have a superintendent who has feeling for town and ex - 
pe«ise in his area. His staff consists of several assistant ^"P';:""^^"^ff 
other special assistants tc aid in the task of organizing and airecting the grow- . 
ing school population (Apper.dix-Table 10) . 

TEACHbliS: The teachers in Fair City are «tr«»«ly .^^^^ ^"^i^f^^^.'^J "^^rv 
abov^ the nat ional statistics in their educational attainment level and in salary 

(Appendix-Table 9). This has been largely due to the ^^"V"* ^ ^t^ir' 
ihi?h has attracted able and dedicated teachers. These teachers tend to be 
creative and innovative because of the recent policy initiated by the 
Zclltol by thrschool board that gives wide latitude to the teachers in their 
approach to subject matter and their management of 5^«^^""Sri;"ii,est struggle 
teachers wield much influence for the good of the children. Their "^^"f^'j 
has bee:, to decrei^se the pupil-teacher ratio from an aver«tge of 28 to 1 to 23 to 1. 
but as yet the cost is too great for such a change. 

SCHOOLS: The school system has grown by almost 72% since 1950 and now 
cont ains 21 g rade schools. 7 junior high schools and 3 senior high schools 
apieiSix-TablesS and 11 . The school year is 39 weeks long, beginning in 
lep^Iibe^ a^d ending the Middle of June. There has been some discussion of 
in?tuuting an extended school year but this plan is only in a preliminary 
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MAP INDICATING THE LOCATION OF SCHOOLS IN FAIR CITY 
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See Appendix 
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JL Junior High School 

▲ Elementary School 
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research stage now, and there are some groups strong I \ opiH)^od to the plan. 

. Elementary schools have self-contained classrooms in which the teacher 
is in charge of all students. A few specialists arc employed for the schools » 
but Bostly for remedial work with slow learners. Recently, teachers have 
been encou]ptoged to vary their programs to meet ipdividual and group needs. 

classrooms were tried in the schools last y«ar, which 

I seemed to work success- 
fully for some of the 
teachers. 

The junior and se- 
Iniot high schools are 
Idepartmentalized and , 
I children are grouped by 
lacademic achievement 
land ability in most 
tareas of study, ^ In the 
Ijunior and senior high 
Iprogram, there are 3 
Igroups of students: (1) 
Ico liege preparatory 
[superior^ (2) college 
preparatory regular and 
(3) occupational prepa- 
ratory. Students are chosen for the superior class on the hasis of (a) grades 
(b) initiative and (c) teacher reco—e n dations. Pupils elect whether they 
are in college preparatory regular or occopatiimal preparatory, tait counsel- 
ing is available at every level to ^ide * 
pupils in making their choice. Approxi- 
mately % of all Fair City pupils are 
classifieii as college preparatory su- 
perior; another 40% ^e regular college 
preparatory. The occupational prepara- 
tory contains 45% of tile pupils and 
serves them well in preparing them for 
occupations immediately upon high school ^ 
graduation. There has been some pres- . 
sure to make the preparation for jobs 
more relevant to Fair City. At the 
present time, negotiations are taking 
place with various companies in Fair 
City to place high school students in 
internship programs for a ^mall portion 
of the' high school occupational program. 
This would mean a great change in the 
educational framework^ since students 
will leave the school, and is therefore 
being given careful consideration* 




IVES HIGH SCHOOL 
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speciir3Ti5i755S?r7T{ some of the elementary junio h ''"^ 

for children with va: .ou. physic.ror "V, .,5' he 'morional- 

rams.fcr the physical ly handicapped, the mentally retarded ' ' 

IV disturbea. There has been 'a small movement to end /fom * 

of the stigma attached to the label, but the Pair City Sch ol Board is 

mitted to their usefullness. 

A smalf Program Headstart began two years ago in F-r City ^"J^e black 

mCKT AND COtWlNPV INmVEHEOT - Fair City has a '^^^^ "[^.^ 

comuHity - involvement and support of the schoo system. In past years y 
have supported, land ana millage issues , the 

heen a SLl;.e .n support ofj-iUage - ^"-^-^ ; t/^hat'^e; ^^^t form 
millage renewal was voted down. The citv scnooib 

a clo!er relationship with the P-^f^^'/^^P^^^, l^,^^"Sr Lum u^^^ be^un a 
complexities have grown rapidly in 3,^00 s and. talk to the 

concerted, effort to encourage parents to visit schoo s 
.teachers and principals. He ^as urged the P.T A to actively ^^^^ 
bers and to make each member a vital part of the school '''^ 
derately successful in the middle class areas, but the lower class areas have 
shown little interest in participating ir. any organized group. 

.n order to further .ood relations the school h rd h,^^^^,^^^^^^ .eHicle 

for community and parent ^"-"Pl^i^JJ^f/^^ legi^ mate are acted upon . 

. ed are considered seriously, and those that are ^^gj^'"; ^^.^^ the parent 

■ Another project -^.^h has proven extremely ^^^^-^^ l^.T^^^ 
-aorer.re':v:rrL:S rrhrnu he 

These teachers' aids are e^t'^e'"^^^,-^^^ '"4'^^"? is engaged in teach- 

remedial help to children while the classroom teacher s eng^g 

ing the .ther children. ^ - hoped tha J- P-f ^^^^ ,,1, ,he .hool. 

include ail the schools in Fair tity. as 

are served by these teachers' aads. 

A p„„ t,ae„ iy the u.;ver;.ty sho.ed that 80 pcr..„t^oJ parents^of ^ 

S t° pJ.nc.pils and Lther administrators -ere s,.,Ur. 

Certa. groups ot ^^.^^^ .^st^aice! S rrcrnr"o? Thf^ 

rr:il1i!^'rtho;:o^U ?:u"i":nrraotton .Uh the preparation the .„i.d- 
ron were receiving in th<.' schools. 
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UTOPIA STATh UMVERSm ; 



The fair city 'Miii(ju> otopia State Univer^it. 
was bulltin 1950 as .^e second largest and one of the -o5t beautiful .ollege 
campuses in the state. It is situated XKong the roUin;; hills bn the edje 
the city, 'in view of the winding North River. It is a .o-educav lonal institu- 
tion with approxiaately 15.000 full-time students. In recent yea/s, more th..ii 
a dUMn new and striking buildings have been <dded to the 32 acre cjwpus. The 




inWIA STATE UNIVERSITY \ 

school prides itself on its high academic standards. Nearly half the faculty 
has doctorates and the student-farulty ratio is 15 to 1. 

U.S.U. initiated a new curriculum in 1967 with emphasis on independent 
study and study initiative, k variety of special programs is offered to meet 
the needs of the students with different abilities and interests. Among these 
progrtas is the junior year abroad, semester in Washington, D.C., and other 
special programs which also allow residents of Fair City to enroll in courses 
of thtir choice for little tuition. 

FAI R CITY CCMtUNITY COLLEGE : Fair City Community College was built in 
1968 I "meet the needs of the industrious and striving citizens of the tovm 
who wanted further education to supplement their skills in their present jobs 
or to prepare themselves for future jobs. Alreat«y, there are 2,500 part-time 
students a-Jd 250 full-time studehts at the college. The teachers are promi- 
nent citizens of Fair City who wish to impart their knowledge to others, iiie 
student-teacher ratio is 20 to 1 and in many cases teacher aids are recruited 
from the gro<ip of former students so that each student can get individual 
attention wh^ he needs it. There are no prerequisites for entering Fan City 
Comwunity College, exce» : the desire to learn. 



The college offers associate degrees in chemical, mechanical and indus- 
trial technology; business, finance and remedial courses; and two-year courses 
in liberal arts and sciences transfer programs. This kind of educational 
facil-vy • \s greatly benefited the economic prosperity of 'he community, its 
new presiaent, Mr. U. Tilitarian, has a close relations>^^ :o the community 
and its needs, and is m constant touch with the inner workings of the school 
and its usefullness to the community. 
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HEALTH & WELFARE 



The Fair C^;. H.„.,„ ,..,.u,.,,< ,»v. ,je. h, ■^^^j:^''::^J^ 

food nroc-ssine and insert control ^tano^iru'^ , ..iiuiua ^ ^ ^ 
rooa proc.bMi). auu .o^r^irc; m the CUV whub. provide medicdi. 

education; and operates several ciiT^ics in int ^.o i i 

dental and laboratory services. 

The County Welfare hep... .ent a.a. . varut^ ^'^'^T.Znt t!^p^t^' 
of ,sistance. Under various government projn-a^r., th. ^-P'^^^'"^"^ i I.d 
blind disabled, dependent cniidren, the aged ana the unemployed. It a us 
o her; xn finding iol.. .n ^ax-ung legal .erv.ces ..nd in -J-^^;; J/ 
housing. The.depa tr,eut is especially concerned witn the f ^"^^ 

providts assitance r, those who are needy by olfering free lunche. a. t.ec 
medical care administered through the schools. 

The health care of its people has always been a prime concern in Fair 
City ^he'cood Samaritan HospxLl has been a symbol of this ^oncer- n . s 
biiU in 1910 as one of the first hospitals xn the state From '-t -e on. 
It has continued to be a leader in innovative techniques and in attracting 
iualiJied staff. Utopia State University has a fine nursing program, which 
has been long cr mected with Good Samaritan. 

the present time, the ^nty has entati..^^ 

a children's hospital in Fair C;;ty which wouiU at a cost 

care here. Care for the poor i3 ^^^^//^^.^^^^"Jf in l-air City keeps accu- 

commensurate with their income. The heltare utr 

rate rolls pcuple m need. 




GOOD SAMARITAiN HOSPITAL 
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GOOD Tl 




Th* residents of Fair City enjoy all of the cultural =^"^""Jf "JJJi.. 
benefit, of ..tropolitwi living. The Fair City Symphony provides the fln.st 
ir ■usic.l entertlinnent and the city's aany parks and playgrounds of far 
space for relaxing for young and old alike. 





VcTRIMBLETOE AUDITORIUM 
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TAEU3 



LOCATtOK 



ATTENDANCE AREA 



ndi SCHOOLS 

iridf • 

-llbMdi HIGH SCHOOLS 

/ " 

i niiiW NcTriableto* 
' mrt tomts 

ELBCI^ARY^ SCHOOLS 

J ; " 

i, J. Hon^r 

tt. D. Doc\ 



L. N. Huffct 
J. Sprat 
Calicocal 
T. ftlinaaicc 

H. &. Diaaic 

0. K. Cole 
ShaftM 

H. M. Contrary 

Inashoe 

S. Siaon 

Mttbard 

Sitpence 

t. B. t. Shaep 

1. •. ilue 



105 
116 
153 



101 
lOS 
111 
122 
115 
1S2 
134 



100 
101 
101 
102 
10$ 
106 
110 
113 

lis 

117 
120 
122 
124 
126 
127 
12S 
131 
132 
133 
I3S 
Rural 



100-109 

110-127, Rural Aroas 
12ft-136 



100-101. 106 

102-105, 107-109 

UO-IU 117.11R. 124-125 

119-123, Rural Araas 

113-116, 126-127 

130*132, 136 

12t.ia9, 133-135 



100 E*st-Cantral 101 

101* 

101 

102-104 
105, 101-109 
106-107 
no-Ill, 119 
113, 116 
M4-11S 
112, 117. 118 
120-121 
122 

123-124 
12S 126 
127 

I2$-I29 
131 

130. 32 

133 134. 136 

I3S 

Rur^il 
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TABLE 4 



XGH SCHOOLS 





ery Dm 
ran Gordon 
Isie Narloy 
oily Flinders 
fiUiw McTriBbletoa 
Robert Barnes 
John Watts 

L. J. Homer 

P. P. P\i^>kinemter 

Knaveofhearts 

Jeckendjlll 

li. D. Dock 

M. Goose 

Laab 

L. M. Muffet 
J. Sprat 
Calicocat 
T. B15.ndAlce 
ii. b. Diddle 
9. K. Cole 
Shaftoe 

M. N. Contrary 
Inashoe 
S. Siaon 
liubard 

Sixpence ^ . 

B. B. B. Sheep 
L. B. Blue 



TOTAL 


NON -WHITE 


PERCENT 


WHITE 


PERCEN 


1554 


758 


48.8% 


796 


51. 2% 


2756 


280 


10.2% 


2476 


89.8% 


1461 


974 


66.7% 

- 


487 


33.3% 


928 


477 


51.4% 


451 


48.6% 


621 


375 


60.4% 


246 


39.6% 


1034 


249 


24.1% 


785 


75.9% 


934 


97 


10.4% 


837 


89.6% 


723 


14 


1.9% 


709 


98.1% 


687 


451 


65.6% 


236 


34.4% 


792 


558 


70.5% 


234 


29.5% 



367 


358 


97.5% 


9 


2.5% 


808 


433 


53.6% 


375 


46.4% 


630 


365 


57. 9« 


265 


42.1% 


786 


595 


75.7% 


191 


24.3% 


6r: 


309 


47.5% 


342 


52.5% 


660 


5 


.8% 


655 


99.2% 


719 


39 


5.4% 


680 


94.6% 


455 


5 


1.1% 


450 


98.9% 


457 


10 


2.2% 


447 


97.8% 


772 


60 


7.8% 


712 


92.2% 


666 


6 


.9%» 


660 


99.1% 


621 


221 


35.6% 


400 


64.4% 


762 


265 


34.8% 


497 


65.2% 


766 


36 


4.7% 


730 


95.3% 


339 


0 


0% 


339 


100% 


662 


150 


22.7% 


512 


77,3% 


616 


347 


56.3% 


269 


43.7% 


554 


337* 


60.8% 


217 


39.2% 


420 


420 


'^ 


0 


0% 


441 


4^*1 . 


. *J0% 


0 


0% 


241 


0 


0% 


241 


100% 
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PUBLIC 


SrH^i .-N.pnil.MFMT RY LEVEL AND RACE, LAST YEAR ■ 










TOTAL 


NON- 
WHITE 


PERCENT WHITE 


PERCENT 








Eleaentary 

(K-6) 


12.393 


4,402 


35.5% 7.991 


64.5% 








Junior High 
(7-9) 


S.719 


2,221 


38.8% 3.498 


61.2% 








Senior High 
(10-12) 

"Total 


5.771 
23,883 


2,012 
8,635 


34.7% 3.759 
36.2% 15.248 


65.3% 
63.8% 










TABLE 6 










PERCENT OP SCHOOL AGE POPULATION ENROLLED 
IN PUBLIC SCHOOLS: LAST YEAR 






-r^ 


TOTAL 


NON- 
WHITE 


WHITE 










Elementary 
Junior High 
Senior High 
Total 


90.4 
89.5 
87.8 
89.5 


96.0% 
94.6% 
90.1% 
94.2% 


87.3% 
81.5% 
86.5% 
86.9% 










. TABLE 7 


r - ENROLLMtK'l 


T^Y SENIOR HIGH SCHOOL UlbiHlLlb: LAbi YtAR 




4IGH SCHOOL AREA 


K-6 


7-9 


10-12 


TOTAL 




Bridge 

Ives 

Banbury 


3.902 
5.798 
2.693 
12.393 


1,549 
2,691 
1,479 
5.719 


1,554 
2,756 
1,461 
5,771 


7,005 
11,245 

5,633 
23,883 


J 



TABLE 8 



F= 


STUDENT- 


-TEACHER RATIO 




.HOOL TYPE 


STUDENTS 


TEACHERS 


RATIO 


Elementary 
Junior High 
Senior High 
Total 


12,393 
5.719 
5.771 

23.883 


413 
204 

'222 
839 


30 
28 

26 

28.5 1 
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'St ATI sties ON CLASSROOM TEACHERS 





NUMBER 


PERCENT 


NATIONAL AVERAGE 1 


Full-Tine Classroon Teachers 


839 




^ i 


Level of Preparation 






« 


No Degree 
B. A. 
M • A • 

M. A. * 30 
Doctorate 


143 

571 

117 
*** 

8 


17.0% 
68.0% 

1 A Aft 
14 . 0\ 

.**« 

] .0% 


7.0 
69.6 

1.3 

.1 


1 eacner ■ u^nover dy \^xm99 








Resignations 

Other (Death, ProBU>tion» 
Ret i resent) 


112 
43 


13.4% 
5.1% 


7.7% 
3.0% 


Total 


155 


18.5% 


10.7% 



•♦♦Included with M.A.'s 

TABLE 10 



SChOOL SYSTB4 STAFF 








...969 








.. .. 1 








3 








. . 2 










4 










.... 31 






Assistant Principals... 




27 










839 






Elementary Teachers 


(413) 








Jr. High Teachers 


(204) 








Sr. High Teachers 


(222) 












... 12 




















9 










6 












. . . . 89 








47 

, 42 
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TABLL 11 

POPULATION DIStRIgUTION BY RACE: 1950-1970 

YEAR TOTAL NON-WHITE PERCENT WHITE PERCENT 

'1970 122,594^ 46,558 38.0% 76,036 62.0% 

1960 112,237 35,018 31.2% 77,219 68.8% 

1950 71,100 14,431' 20.3% 56,669 79.7% 





< 


TABLE 12 


\ 

\ 


ECONOMIC CHARACTERISTICS OF THE POPULATION: 1970 


Total Personal Incoae 


$287,775,576 


Per Capita Personal Income 


2,348 


Per Capita Effective Buying Income 


2,008 


Household Income (Average) 


6,943 


Household Effective Buying Income 


5,775 


Per Capita Total Local Taxes 


143 


Per Capita Taxes as Percent of 
Per Capita Income 


6.1% 


Percent of Unemployment 


6.5% 

- 



TABLE 13 



[ ■ 


POPUUTION MOBILITY: 


1960 TO 1970 








TOTAL 


PERCENT 


NON- 
WHITE 


TEKCENT 


WHITt 


PERCENT 


Persons S yjs. 
& older 


108,585 


(100) 


t 

42,777 


tlOO) 


65,808 


(1003 ! 


Reside in sane 
house since 
1960 


56,463 


52% 


18,822 


4^% 


40,143 


61% 


4oved since 
1960 


32,122 


48% 


23,955 


56% 


25,665 


39% 


4oved within 
city 


39,091 


36% 


14.972 


35% 


17,768 


27% 


Moved to city 
from outsiae 


13,031 


12% 


8,983 


21% 


7,897 


12% 



66 



GAME MANAGER'S MANUAL: SECTION II- 
CHAPTER III . 



FEHR-PRACTICUM 



Players Instructions 

© 



r V 



FUNDED BY 

U.S. OFFICE OF EDUCATION 
PROJECT NO. 0-9047 
CONTRACT NO'S. 0EC-0-70-4773(520) 
0 OEG-0-72-0529 



LEVERNE S. fOLLET 

421 5A SCHOOL OF EDUCATION BUILDING 
UNIVERSITY OF MICHIGAN 



b / 
- 70 - 



GAME MANAUlR'S notes FOR PLAYERS' INSTRUCTIONS! PREFACE . 



This preface does rrot appear in the Pla^yers' Instructions booklet 
used by the teams. In addition, because of trie complex nature of some of 
the instructions, it was necessary to interleave explanatory notes to the 
Game Hanager throughout the booklet. To clearly differentiate the notes 
from Instructions, they are pUced on facing pages. Beginning overleaf, 
a page from"the Players' Instructions booklet will. appear on each right-hand 
page and the corresponding Game fnanager's nbtes on the left-hand facing page. 
For additional clarity, appropriate titles are placed at the top of each 
page- 

Preparing for a Session ■ . 

The placers' instructions provide step-bj;^-step directions for completing 
the work assigned in the practicum. At each step, various optional tasks- 
(assignments) of varying difficulty and complexity are provided. The Game ■ 
Manager's major preparation task is to choose the set of tasks to be assigned 
for the session at hand.X In the Game Manager's- notes we will point out the 
options and provide sDme^general guidelines for choosing among them.. However, 
the Game Hanager must decide which options best suit his/her particular com-- 
bi nation of;instruct1onal objectives, trainee entry skills, and available 
local resources. 

It is essential that the Game Manager become thoroughly faalUar with 
the players' tasks at each step of the practicum before attempting to choose 
among the optional assignments. Since both the conceptual difficulty and 
the time requireo for the various tasks changes radically from problem to 
problem, the only safe procedure is for the Game Manager to walk- through the 
entire practicum step-'^y-step with each problem (RFP document) which- is to . 
be used. It is highly desirable, but perhaps not essential, that all con- 
sultants have direct experience with tlie particular problem or problems 
being used. Section III provides specific Erections for a walk- through 
orientation to the standard REMAR problem. Section IV provides suggestions 
for adapting the orientation to the other seven problems. 

There are, of course, a number of preparatory activities related to 
the cnoice of researcn consultants, arrangements for suitable physical fac- 
ilities, and acquiring ample FEHR materials and auxiliary references. A de- 
tailed discussion of these matters appears in section III. 

The First Meeting 

The Players' Instructions begin w^ith the assumption that each prospec- 
tive player (trainee) has read the introductory materials presented in Sec- 
tions I and II of this manual. It is suggested that these materials be 
passed (or mailed) to each player, at least one day prior to the first for- 
mal practicum meeting, as a formal reading assignment. At the first formal 
meeting, it is suggested that the Ga-^e Manager might proceed as follows: 

(1) Provide a brief overview of the goals of the practicum session 
aAd a general description of the activities in which players 
will participate 
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m& MANAGER'S NOTES KOR PLAYERS' INSTRUCTIONS. PREFACE 



(?) Briefly review the nature of FEHR-PRACTICUM using the rhart on 
page 4 of the Players' Introduction to FEHR-PRACTICUM ^Section I 
of the manual) as a guide. 

(3) Pass out the RFP document to be used during the session. 

(4) Read the first two pages of the RFP aloud to the class. 

(5) Point out +he list of tasks on pages 2-5, s? that these will 
be discuss ' fully next day. 

(6) Discuss briefly the contents ana "use of each appendix. It is 
suggested that you: 

(a) Display the Informatiou Bank and explain how to use it. 

(b) Illustrate the use of the Research Population in Appendix II. 

(c) Briefly discuss Appendix III pointing out the various treat- 
ment costs> emphasizing the nature of the time units* and the 
bottom of tne Dage without d'^tailed expU lation. Explain how 
to rfead the catalog charts. 

(d) Discuss the Catalog of Variables in some detail. Read aloud 
the iiotes on page 1, and ooint out the costs listed at the 
bottom of the pa^e without detailed explanation, ^xplain how 
to read the catalog charts. 

(e) kead aloud those sections of the remaihing appendices which 
you judge to be relevant to the completion of the Orientation 
Questionnaire. 

(f; Pass out the Players I.istructions booklet and assign Task 1, 
next page. The remainder of these notes are synchronized with 
the iristrjctions and the corresponding items on the Checklist 
of Tasks to be Performed from pages 3-6 of the RFP document. 
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GAME MANAGER'S ^TFS FOR PLAYERS' INSTRUCTIONS: PAGE 1 



TASK-l Qrientatin uu.otio maire . Much of the advantage of the FEHR system 
derives from the persona! involvei nent of each player in "solving the proDlem.. 
It is important to establish this involvement very earW in a session. Con- 
sequently, we reconnend that each individual player brf required to complete 
an Orientation Questionnaire prior to the formation f teMJs For this 
reason Item one on tne Checklists of Tasks (see pagi 3 of the RFP docu^4nt 
for problem concerned) is ii«rked with an asterisk. 

It is suggested that ihe Game Manager: 

(1) Reaa aloud the instructions f'"- Task 1. 

(2) Briefly discuss the contents of the Orientation Questionnaire. 

(3) Require each player to complete the Orientation Questionnaire 
independently. We suggest that placers be supplied with an extra 
copy of the questionnaire rather than completing the one in 
their Instructions booklet. (Extra copies can b^ ordered sej)^ 
arately.) ' 

(4) Try to encourage players to answer all itsras thoughtfully and 
completely without making the task appear prohibitive y difficult. 
It is neither necessary nor desirable for pWers to bone up 

on the literature (Information Bank) or research methodology 
before answering the questionnaire - its sole purpose Is to 
motivate careful analytical reflection at the player's present 
level of expertise. 

(5) The questionnaire might be finished during the first meating. 
if time pemits, or assigned as 'homework". In either case, 
remember to allow time for reading the questionnaires before 
fomring teams. 

(6) Before dismissing the first meeting, it is a good idea to pre- 
view the contents of task 2 sp that players will know what to 
expect. In fact, players should be encouraged to read through 
all the instructions. ■ 

TASK 2 Organi zation and Assignments . Tl^is tasP would normally De completed 
durinq the second meeting. l2o thing s must be a complished. H) the players 
mSsi be organized into tlams. and (25 th8\ tasks to be comp^et d Juring his 
pfrticular session must be assigned. The \assiQned tasks should be checked off 
?n each Slayers checklist (page 3-5 of the\RFP). This may be done at the 
Sco or!'to saveMe. each list can be checked off prior to being 

distributed Use the Game Manager's notes Associated with each of the tasKs 
ti h^lp you decide whether or n?t to assign the task and to decide among options 
within each task. 

The options within task 2 involve salarV ^^'^/^^i^'/^^^.'ftf;?^ °^ 
questing infonnation from the data generator, and a log of activities, borne 
conments and suggestions: 



(contitiued on oaae 75} 
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PLAYERS' INSTRUCTIONS PAGE 1 



INSTRUCTIONS 



The intent of this package is to provide detailed instructions on how to 
fulfill the tuSks Whicti have been assigned to your team. At this point >ou 
Should oe familiar with the background information contained in the Player's 
Introduction to FEHR^PRACTICUM. and Fai r City, U.S.A .. and have read the first 
two ^ages of your Request for Proposal and scanned the Appendices. These in- 
structions correspond to the list of tasks (pages 3-6) in the RFP. It might be 
helpful to remember that these tasks are somewhat progressive so that you may 
find that the later .^^sks are mere elaborations and extensions of some of the 
earlier tasks. 

Task 1 : Orientation Questionnaire 

This questionnaire, which appears on the fol lowing, page, is intended to 
guide you to those aspects of the game which you should be considering. The 
responses which you give will not be used to evaluate you, although the game 
manager may use them to form effective teams. Please make your responses as 
concise and specific as possible; the length of each shoula be determined by 
the space allotted to it on the questionnaire form. Your game manager will 
tell you whether you are to use tSe ropy in this booklet, cr if he/she will 
provide an extra copy for you to use. 

Task 2 : Organization a nd A ssignments 

At this ti"ie the game manager will give directions on forming teams. He 

will also infona you what the team salary is for your particular research, and 

will exolai, ..ow the simulated costs will be handled. Further, he will give in- 
structions on how to use the computer. 

The tangible work which you or your team must produce is determined by 
the specific tasks which the game manager has chosen and these should be checked 
off on the list of tasks contained in the problem description. These tasks will 
be described more fully as is appropriate. However, the game manager should give 
you a fairly clear schedule of events at this Mme. 

One of the tasks you may be asked to perform is to keep a log. Research 
in -education, like an experiment in chemistry, must be carefully recorded in 
order to accoirnt as well .as possible for all influences on the experiment or 
research. Consequently, it is useful to the researcher and interested parties 
to keep d log containing every relevant detail of procedure and process as a 
precise history of the research. It is likely that in the latter stages of your 
research you will wonder why you took various courses of action. Therefore, 
even if a log is not a requirement, it is a highly recommended tourse to follow. 

Ewtries into the log should be brief and need not be made for every work 
session. The type of information which you might include in the log would be: 
determination of a research strategy, and any changes in this strategy; use of 
the computer for generation of FEHR data and for data analys.s; and use of FEHR 
resources (consultants, I.S.T. uints, references, and Information BankK 
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(1) Team salary is' applicable whenever costs are used. It has 
provan to be a very useful device to motivate teams to p3 an 
their activities and budget their time thoughtfully. However, 
the salary item must be a significant portion of the total bud- 
get to be effective, the i afore oifferent salaries should be used 
witn each problem. Suggested levels for each problem are given 
in Appendix I. High-r rates than those suggested may be usfd 

if it seems desirable to increase the emphasis on completing 
tasks as scheduled. Emphasize that salary is paid on real 
time and not on simulated FEHR time. Note, too, that the same 
salary is used for all teams ev^n though they are of different 
sizes. 

(2) In our 'ield te-ts, costs have proven very useful in creatina a 
"real -life" atmosphere. We recommend that they be used if time 
permits. However, if there is insufficient time to calculate 
research budget, costs may be ignored. 

(3) The normal method of paying for FEHR costs is by checks (option 
b). However, if your computer installation has sufficient disk 
space available, it m^ be possible to adapt the FEHR automatic 
accounting program used at Kichigdn to your system. Ho*«vcr, 
this program necessarily depends on a variet;y d system routines 
which differ from installation to installation. You would re- 
quir a systems programmer at your installation to do the actual 
adaptation. Further inf'^rmation is avail able from tht authors 
on request. 

(4) Keeping a log is a relatively easy task which supplies a great 
ueal of useful evaluative information. If individual perfcrm- 
ance is to be assessed, each player should keep a log. 0th sr- 
wise, one log may be kept for each team. 
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ORIENTATICN QUESTIC:iNAIRE 
^1. What observed problem prompted the sponsor to put out a reauest for proposals? 

2. What would convince the sponsor the problem has been solved? 

3. What modification of the sponsors' view, if any. would you m^ke? What argument 
would you use to convince the sponso.s to alter their views? 

4. Considering all the views abov-, define the research problem as you see it. 



5 In making an experimental coinparison. what classes of variab es would you use to 
measure the effectiveness of the various treatments (programs available in this 
problem, i.e., IQ, achievement, perceptual, personality, etc.? 



Assuming you could not use more than a total of 100 subjects in the ^'^^^'"f . 
woulS you go about selecting this group from the total available research popula.. ,n? 



7. Briefly describe how vou would divide this group of 100 into ths available treatments, 



8. Briefly describe the nossible techniques you might use tu analyze the data collected 
after the treatments nave been administered. 



9. How does this p.-oblem's content area relate to your educational interests? 
10. Briefly describe what past experiences you have had with research. 



GAME MANAGER'S NOTES FOR PLAYcRS' INSTRUCTIONS: PAGE 3 



TASK 3 Qnerationa rizinq tue Problem . On first reading, part A of this task 
m^y appear deceptively simple, ^ince each player was required to define the 
problem informally in the Orientation Questionnaire, it appears that a br ef 
discussion is all that is required to produce consensus. It is our experience, 
however, that substantive differences in team members' views almost always 
appear when the team begins to select the specific dependent variables to 
be used ant* set the criterion levels which indicate successful achievement. 
Such conflicts provide a powerful motivation for a cr- -al review gf 
the research and an intensive study of measurement theory as it relates to 
the particular tests available. In sessions emrfiaslzing problem-definuion 
skills, we recomnend "seeuing" the teams tc enrfifre a varleiy of viewpoints. 
For example, each team might contain an administrator, a researcher, and a 
teacher-educator (hopefully of clinical orientation). 

The Checklist of tasks provides optional assignments with respect to 
both the amount of background reading (review of the literature) and the 
problem definition fier se. Corments and suggestions; 

(1) If the players have a substantive Interest in the contartarea 
of the problem, they should probably be asked to review the ori- 
ginal articles in their entirety rath«r than just the Information 
Bank sunnaries. In this case the Bank summaries provide a se 
ful role in determining which studies might safely be ignored. 

It has been our experience that team members interested in the 
area frequently do fairly extensive library searches even when 
they are not assigned. The "Information Bank only" option would 
normally be assigned if the content area of the problem was not 
of direct Interest to the players. 

(2) If there are stringent t^me constraints or if the session is to j 
be devoted to research design/data analysis, it may be desirable 
for the Game Manager to provide an operational statement of the 
problem. It is our experience, however, that most clitnts are 
extremely weak in problem-definition skills. We recomfwrJ stude.it 
p?-t1cipation in problem definiti'*'-. wherever possible. 

(3) The problem definition form is intended to push players beyond 
the simplistic immediate problem stated in the RFP to the basic 
problem (or root problem) which the prograins being evaluated are 
attempting to address. Specific rotes are provided for each 
problem area in Section IV. 
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Task 3 : Qperational izi ng the Proble m 

After you have formed -.ntj teams, you shoulc work toward conceptualizing 
the problem. Remember that the preliminary parts of research involve a con- 
tinuing process of clarification, re-evaluation, and integration of new facts. 

The following items are intended to help you clarify your thinking about 
this problem: 

A^ When your team first gets together, compare and contrast your in- 
dividual problem definitions, based upon the Orientation Questionnaire. 
You should try to reach a problem definition you can all agree upon. 

b. Review the literature on the content area and reconsider the problem 
definitior. as seems appropriate. 

C. Formal problem definitipn: Based on your collective views and the 
literature, you are to arrive at a definition of the problem. The 
process by which you may arrive at this definition is illustrated 
by the chart on the next page. The following instructions will assist 
you in cympleting each section of the sheet. Your game manager will 
Vt^. indicate whether you are to use the form in the booklet or obtain 

another copy from him. 

1. A problem may be defined by tne discrepancy between what should 
be and what is ha'?r>ening. ^ 

(aj What sh ould be happening, but is not? This may be stated m 



temis"^f the sponsor* s objectives and your own. 
(b) What is^ happening that should not? 

1. Do you believe your views are different from the sponsor *s views? 
If so, how? How will you handle this difference if there is one?y 

3. Consider the availab^.e treatments carefully. This section of the 
chdrt is intended to help you delineate the differences between 
the treatments. Is there any discrepancy between the problem 
definition you nave and the kind of things these treatments are 
intended to do? If so, re-evaluate. 

4. Wh<jt k.ind of information would you like to have about your subjects? 
Two of the major categories of variables are independent and dependent 
varlaoles. Dependent variables are those which can be influenced by 
treatments. Independent variables are those which influence tne 
effects of d treatineynt. Independent variables can be broken down 
into two types: manipulable and nonmampulable. The nonmanipulable 
variables are those wh! h the researcher can not change; tor example, 
sex. Manipulable variables are those external factors such ds the 
kind of school which tne researcher can have some effect uf.'or.. It 

IS pcssible thrit one variable can be both dependent and independent. 
For example, if you select a student on the bas's of IQ, you can 
also measure the effect o^ the treatment by determining how much 
the IQ scores have changed. You should determine what are the in- 
dependent and dependent variables which you would like to use. 
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PROBLEM DEFINITION FORM 



K (a) What should be happening? 



(d) What IS happening? 



Z. Difference in Views 



3. Treatments 



De siraDle Consequences 



Undesirable Consequences 



4. Independent Variables 

Manlpulable Non-Manipulable 
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Dependent Variables 
Change 



GAME MANAGER'S NOTES FOR PLAYERS* INSTRUCTIONS: PAGE 5 



TASK 4 Determining the Extent and Severity of the Problem , If thare are 
severe time constraints in a session. Task 4 may be skipped in its ent'^rety. 
In thdt case, the Game Manager would normally supply the teams with bo^h a 
problem definition (Task 3) and the proportions of subjects affected at var- 
ious levels cf severity (according to the definition) In each unit and sub- 
units (e.g., schools and classrooms) to be used. If this procedure is 
adoped, the Game Manager must obtain the required information by running 
survey using his problem definition prior to the practlcum session. 

.. . A useful ^alternative to supplying the extent a^ severity information 
Is to gather the Information collectively with all teams completing task 4 
as a single group. This has the additional advantage of familiarizing 
pliers with the procedures for requesting Information from the data genera- 
tor. However, if time permits,' we recommend that each team be r^uired to 
design and run its own file searches and/or surveys. If a practical demon- 
stration of a file search/survey seems desirable, a sample request using a 
few randomly selected subjects and a variety of (perhaps irrelevant) vari- 
ables/ The validation request listed In Appendix I» (or a modification 
thereof) may be used for this purpose. 

The comments and suggestions for each sub-task are listed below: 

Sub->task 4.1 Submit a Request for a Planning Grant. 

(1) If time is constrained, the budget aspects of task 4 mi^ be ig-* 
nored. (Hon^ver, see coomnt 3» below). 

(2) The suggested maximum grant for each probiM appears in Appen* 

'dix K 

(3) Me have found that the needs of the teans varies so greatly that 

a maximum grant is of limited value in determining the level which 
should be alloted to ar\y one team raarvy teams can and should 
get far less than the maximum. We recommend that awards be neg- 
otiated at a meeting of the team and the Game Manager (or his 
designate). The team should be prepared to justify each item 
on the budoet. Most teams in our filled studies have considered 
this negotiation experience invaluable. 
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' 5. Next, go to the Catulog of Variabieb and i>clect the variables 
whicli you think important. You will notice there is often a 
V duplication of tests measuring toe same thing. Look at the 

prices listed and the descriptions presented in the Information 
Bank. To help you choose among tests compare the prices,-^ 
reliabilities, and test validity stores. You might also dis- 
cover that some variables which you think are important are not 
availaDle; consequently, you might have to alter youjr problem 
definition somewhat. 

' if at this point you or your team remains 'confused about what is 
involveu in defining the problem, you should hire a consultant. 
If you have any questions concerning tne use of FEHR resources, 
. or other technical details, consult your game manager. 

Task 4 : Determining the Extent and Severity of the Problem 

The information vou have been working with so far is based ^n research 
reported in the literature and your own logical analysis Of t^)e prt)blem. At 
this stage you may wish to obtain more precise information about the status of 
the particular research subjects availaole inr your problem by ODtaintng the 
scores of a- representative sample of subjects on the variables (tests) which 
were identified in Task. 3. Th^s can be accomplished by running a File -Search oi 
Survey or DOth. 

There are three main sub tasks within Task 4. These are descrioed below 
as Tasks 4.1 to 4.3. . . 

Task 4.1 : Request for a Planning Grant 

If you are responsible for budgeting funds, it will be 
necessary to obtain a planning grant before proceeding 
"witn your file search and/or survey. Tne first step in 
ttiis process is to fill out a Request for a Plannin g 
Grant form. A copy of the request fom appears on che 
foTTowing page. Your Game Manager wi^ tell you whether 
you are to use the copy in this book, or if he/she^>/iH 
provide an extra copy for you to '\ise. N 

The first five ^ tems on the form can be filled out 
Dy summarizing in a few succinct sentences the information, 
from your Problem Definition Form (Task 3). The 1ST Unit 
on,sat;ipling may be of help in completing item 5. 

• ' Item 6 requests a plan for analyzing your data if 

your team consists entirely of research novices. You 
may have to ao some independent study before completing 
this itenu Your Game Manager will provide a list of the 
independent stud^ materials which are available to you- 

Item. 7 requests a brief itemized budget. All the 
information necessary for preparing the budget is given 
in Appendix IV of your RFP, the Catalog of Variables. 
• Pay particular attention to the general costs given on 

page 1 of t-liis dppendix. Th€ overhead cost for initiating 
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Sub-U&k 4.2 Running a file search or survey. 

(1) Make sure the 'p}i^er% understand the difference oetween file 
searches and surveys (see page 10. Players' Introduction ) and 
the cost of each (see instrjctions pages b and /;. 

(2) The maximum number of variables which can be printed by the com- 
puter on any one file search, surveyed or treatment Is 20. 

(3) The use of restrictions on both subjects and variables Is en- 
couraged. In most problems about 120 subjects will give ample 
statistical power, and their selection will prov i :e trainees 
with vital experience in sampling. In no case should a stu4ent 
be allowed to survey the whole population — It is too expensive 
In terms of -omputer time as well as being experimentally in- 
efficient. In the case of variable restrictions there are two 
considerations. First, a team ought to be aware that several 
tests must be used to measure the degree to which objectives 

are being met. Second, no team should be permitted to define 
the problem in too-simplistic terms. They should be encouraged 
to use designs which stretch their technological expertise. 

(4) Aaditional comnents appear opposite pages 14-18 of the instruc- 
tions which describe the proce ire for preparing a request to 
the data generator. 

Sub- task 4.3 Analyze the Data from Step 4.2. 

(1) Describe the facilities available at your local computer center 
and the procedures for using them. 

(2) The general assignment of files or punch cards in Task 2 should 
be made on the basis of the facilities available. 

(3) , Time and budget allotments must allow for local turn around time. 

(4) The complexity of research design may be limited by the anajjf^^c 
' programs available locally. (However, see 1ST units I and 11, . 

(5) Most canned data analysis programs require tne preparation of a 
format card. Instructions in preparing format cards for FtHK 
data are given in 1ST unit V. 

Sub-tasK 4.^ Formal Problem Description (first draft). 

(1) It is a good idea to require a written problem dpscription when- 
ever a proposal has been assigned. 

(2) In our experience, the critiquing of each team's problem des- 
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FEHR-PRACTICUM 

RtQUEST FOR A PLANNING GRANT 
TO SUPPORT A FILE SEARCH AND/OR SURVEY 



Team Date Amount Approved by 

Requested (Gams Manager) 



1. 


State' the problem In operational terms. 


2. 


Briefly explain how you plan to use the requested funds and what you hope to find. 


3. 


List the variables (tests) which >ou are going to measure. 


4. 


Kow did you choose the particular variables listed above? 


5. 


How many research subjects do you plan to use, *3nd how will they be selected? 


6. 


How do you plan to analyze the data? 


7. 


Itemize the FEHR-PRACTICUM costs of this planning project. ^ 
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cription by the total group usually nlsulu in numerous construc- 
tive criticisms of each team's work. We recommend this proceoure 
wherever possib'le. 

(3) Although a consensus definition is not a necessary cood of the 
critique meeting, it is a frequent result. When consensus oc- 
curs, it may be desirable to conserve computer funds by complet- 
ing task 5 as a cooperative large-group ende. vori If a compari* 
son of experimental quality is to be m? in tasic 7, it is de- 
sirable to get agreement on the dependei.t variables to be used 
and the relative importance pf each (see 1ST unit I). 

(4) For sessions concentrating on problem description this is the 
final task. In this case it may be desirable to expand the con- 
tents of the description to include more details about the sur- 
vey and its results. 

TASK 5 Identifying a Pool of Research Subjects for Treatments 

(1) If all subjects "eligible" for the treatment have similar char- 
^acteri sties, it is convenient to use the FEHR secretary, bee 

; 1ST unit V for instructions. 

(2) The secretary is not an efficient wy to do stratified random 
sampling. It is better to run a survey to obtain the score val- 
ues for all subjects on the stratifying varlable(s). then sort 
the subject-cards (or file lines) using .the system services sup- 

, plied at your computer installation. • 

(3) Tf task fis assigned, great care must be taken to ensure that 
the selection criteria are not too stringent to provide ample 
researc . ^ubjectj. When the FEHR secretary is to be used, it is 
a good practice to insist that a teai^ must be able to demon- 
strate from survey results that they will get the required num- 
ber of subjects before allowing them to proceed. If an ordinary 
survey is used, the selection criteria can be modified after the 
fact without engendering acditional computer costs. 

(4) It is usually advisable to suggest that the surveys be conducted 
on intact units or sub-units (chosen either systematically or 
randomly) — surveying the entire research populativfv is almost 
always prohibitively expensive. 

^ TASK 6 Empirical Evaluation of the Educational Treatments. Comments 
and suggestions are grouped within sub-tasks. 

/ 

Sub- task 6.1 Proposal * ' 

(1) If task 4.4 (Formal problem description) was assigned, point out 
that the proposal ii just an expansion of that material. 

(2) The proposed maximum budget is given in Appendix I. , 

« 

(3) Again it is recommended that different -levels of funding be a- 
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a file search or survey intended to simulate the cost setting 
Up. A change is made for each separate computer run. 

The overhead for entering a unit or a sub-unit is intended to 
simulate costs of interrupting an educational unit (e.g., school) or 
sub-unit (e.g., class). This charge is made for each separate inter- 
ruption. If, for example, you chose 10 students from class 1, then 
10 students from class 2, then an additional 5 students from class 1, 
you would be charged three sub-unit overhead costs: two for class 1 
and one for class 2. 

The cost of variables are given on a per score basis. The price 
of file search information is always five cents per score. However, 
the cost of each survey score dapends upon which test is used: these 
costs are listed on the Catalog of Variables. 

Include a salary iteffi based on your best estimate of the length 
of time it will take to complete the task. 

Task 4.2 : Running a File Search or Survey 

To obtain your file search or survey uata, you must prepare a request 
for tiie FEHR data generator. Complete instructions are given In Ap- 
pendix III of these in'^tructions. 

Task 4.3 : Analyzing Your Data 

Once the data from Task 4.2 are obtained, you msy proceed with the 
analysis. Your Gan^ Manager will explain the local facilities. 

Task 4.4 : Prepare a Formal Problem Description , (first draft). 



A formal problem description is always contained in a research propos- 
al. The contents expected here are describe^ in sections B, C, and D 
under the proposal (Task 6.1). Be prepared to present your Descrip- 
tion for critical discussion by members of the other teams at the time 
indicated on your Checklist (page 4). If the criticisms warrant it, 
revise your description before proceeding. 

Taks 5 : Identifying a Pool Research Subjects for Treatments 

In some problems it is advantageous to run a second s- ^^vey and/or file 
se^ ch to identify a set of subjects who are "eligible" for the educational treat- 
ments to bo evaluated. For example, a remedial program would probably be aimed at 
a snail sub-set of students who were having specific problems. Complete this tasK 
by following the steps outlined for Task 4. 

Task 6 : Empirical Evaluation of the Educational Treatment(s ) 

Task 6 is the major activity of FEHR-PRACTICUM. It consists of four se- 
quential tasks, with each task in itself constituting a complex process. The four 
tasks are describee below as Tasks 6.1 to 6.4. 

Task 6.1 : Proposal 

Submit a research proposal on the date indicated and negotiate the lev- 
el of fundir.g with the Game Manager. The proposal title and the authors 
names should appear on the firs^ page. A table of contents should fol- 
low the abstrict for longer proposals. For more detailed instructions, 
see th'.* recommended references on page 10. 
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warded to each team. The team should be reqii'^: to justify 
each item in their budget. 

(4) Underbudgeting is, in our experience, a more frequent error than 
overbudgeting. 

(5) The choice of writing style should be made on the basis of the 
^ tiL avlilable and the needs of the players, time is con- 
strained, we recommend a Drief outline proposal rather than re 
quiring no proposal for two rpasons: fif^t, we have Jjund that 
the quality of plannin-i -.-.cr '*-.es sign1fici.ntly when the plans 
must be specified in written form, and, second, th^ plroposai 
provides a written record of a team's accompli srunent. une 
same argunents would, c? courst. apply to. the final report.; 

Sub- task 6.i Running the experiment in the FEHR data generator. 

(1) See the cownents opposite the instructions fo»- running a treat- 
ment in the data generator (pages 18 & la). 

(2) The choice of data output depends on your l<*cal computer in- 
stallation. The "printed copy only'^ option would be useo if 
statistical analysis was to be done by hand. The typ« oj. f0P> ^ 
is controlled uy the first control C5rd in a "^fl^^^t to the data 
nA.«rator, see the Gane Manager's notas oppostie page 14 of the 
Instructions. 
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A. Abs t ract- 'This snould be an ubjertive, dccurate, clear 
suninary of the essential infomiation contained in tne 
proposal, including objectives or nature of the project, 
a practical justification for this research, and pro- 
cedures to be used. 

^- Introduction and Overview of the Problem --This should in- 
cluc*.- a statement of the problem or purposes in terms of 
background information pertaining both to this school 
system and other research in this field. 



C. Review of Related Li terature- -Sorr.e indication should be 
given of how the proposed research will extend, refine 
or revise existing knowledge. A uriefly annotated bib- 
liography should be included. 

Statement of the Problem in Operational Terms- -This should 
be in the form of a hypothesis concerning the relevant 
variables, the relationship between them, and the target 
population. 

E. Pr ocedures --This section should describe the steps to r^e 
used in testing the hypothesis, including sampling methods, 
research design and rationale, treatments to be adfninistered, 
variables to be considered, data collection, and data 
analysis , **ocedures. 

Schedule of Activities- -Th should indicate the steps by 
which you will progress through the above procedures, as 
well as the use of the Information Bank, 1ST units, and 
any other research which you plan to do. A flow chart 
may be usee 

G. Buuget' -Your budget should state exactly how much money 
you are requesting, and specify how these funds are to be 
spent. These costs should agree with the procedures whicn 
you outlined above, although yc « should allow for unplanned 
expenses. The areas which should be included are: terr. 
(time), salary of team, consultant fees, file searcn ana 
survey costs, and contingency funds. This must be carefully 
itemized. 

Task 6.2: Collect the Uat a 

After dividing your total sample into -treatment groups, run 
your experiment via the FEHR Data Generator. See Appendix 
III and/or your Game Manager for instructions on how to 
enter your request and obtain your results. 

"Research may be one path tv. many destinations or 
many pduis to One oes ti nati on"-*FPS 
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Suft-task 6.3 Analyze and interpret the data (from 6.2) 

(1) See coirroents for sub- task 4.3. 

Sub- task 6.4 Submit a final report 

(1) Teams will probably need help in deciding on the report fcnn c — 
especially tabular material. A good reference should be avail- 
able. (For suggestions see the comments opposite the list of 
references on pages 10 - 13 of the Instructions). 

TASK 7 Attend a Consolidation Session 

(1) The ii»*etir.g should be attended by all teams participating in 
the consultants. If more than one problem (RFP) was used, a 
separate consolidation session should be scheduled for each problem 
area. 

(2) It is advantageous to have copies of the reports distributed in 
advance of the meeting. A less desirable alternative is to have 
each team make a verbal report at the meeting. The latter, of 
course, requires much more time. 

(3) The effectiveness of es h decision should be assessed b. running 
the FEHR Decision subroutine, instructions for running uiis pro- 
gram appear on page of this manual. 

(4) Each team's report should be criti-qued by the Game Manager and/or 
the consultants — preferably In writing. Wherever possible, 
the criticisms should be related to the Decision outcomes above. 

(5) At the critique session, each team report and the associated 
critique should be presented, followed by a general discussion 
of t^8 meriU and de Brits of the various reports. 
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Task 6,3 : An^l^ze "Me Data 

Your Game Manager can inform you about the facilities, 
canned programs, data manipulation procedures, ard types 
of outpu^ available to you. You shoul'i be able to di - 
tinguish between the statistical conclusions of your 
analysis ^nd the educational implications of th03e findings, 
and are urged to keep these conclusions entirely separate. 

Task 6 .4: Prepare a Final Report 

T:ie final report should contain your proposal and all items 
checked off on the list of tasks. It is in this report that 
your team describes what you discovered and what you think 
it means and draw your conclusi'ns. This is, in effect, 
where you put all your work tog :her, where it culminates, 
so make the best of it. 



Task 7 : Attend the Consol ioation Session 

At this meeting you will obtain feedback on the results of using your 
recomnPHded decision for an entire year, and have an opportunity to discuss 
resuUs. In addition, you will be given constructive criticism on your fina 



report. 
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MANAGER'S NOTES FOR PL/iYERS' INSTRUCTIONS: PAGE 10 



K esaarch Consultants 
(1) 



The persons chcsen as research consultants can have a crucial 

effect on the amount learned by the Pl||ye;;:^»-a^2*S'.n ^?ed 
search-training institution ^ ^-^conrond the use of ao _ .red 
students for consultlve" purposes. The advanced students can 
then consult with a single expert (the professor) "PJ*^^*^'^ 
difficult problems. In many settings institutiona credit can 
be Arranged as (real life) "payment" for the consultant . In 
our expedience the consult! ve role has P»-°^«J^J;,'»?ji:;^5es 
rewarii.ig learning experience for the consultants th«?s«'^": . 
ImI procedure has the additional advantage o\P^. J;"^ a h gh 
■ level of Individualized instruction t" player- trainees at a low 
cost. , 

(2) «e have had some negative experience. **jth the requirement thjt 
consultants be included in each team's FEHR J^J^t. Teams were 
loathe to spend mon^y^even FEHR money) °"Jtudent consultants. 
To.awoid this difficulty, we regular y awarded a ^Pecial FEHR 
grant (via the Message Generator) which could be spent oiilx to 
hire consultants. 

* (3) For your convenience, a special from for ^Jtalnlng Vita Informa- 
tion of FEHg research consultants is P'^o^ «*ed 1" i»*!„^fi; be 
this manual. Copies of the vita ^formation would normally be 
presented to each team w.t the problem (RFP) packages. 

I.S.I. Units and References 

M\ ThP materials included under the reference title on page 10 

Ire ~ailab e S paperback, individually bound versions. In our 
field trails these^re used as I.S.T. unitsy th considerable 
Ic^ss: Play^Xfnees found the booklets J" the Proposa 
the report and simplified designs expecially helpful. However, 
they are listed separately because they are not past of the FEHR 
project £er se. 

(2) we reco«».en. that a wide variety of r"«t!;^/,::!!?^;"?,^%e?-' 
awailable to the player- trainees during the session, ine ret 
Irlnces listed on^p^ges 12 and 13 of the PlayeS Instructions- 
would be suitable for 1ST purposes. 

(3) In our Initial FEHR writings. InJ^t^^lJ^^nk materia^^^^^ 
I3T units were budgeted items with per- tern costs attached. 
This was abandoned when a per diem (real ^i-ne) team salary was 
adopted. The rationale was that any time spent using these ma 
terlals would now be charged through the (real-tliw) salary. 
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APPENOU i 
FEHR RESOURCES 



RESEARCH CONSULTAN TS 

The rescfarch consultants in the FEHR-PRACTICUM game ^e'-.e the same function 
as th-ey would in real -life research. You may hire a consul lant to advise you 
.whenever you are uncertain about some aspect ycur research methodology. 

Each consultant's fee and schedule are located in the upper right-hand corner 
of his vita. To hire a research consultant, you must enter your request on the 
"Consultant Waiting List" through the game manager. Since a number of teams may 
be playing a FEHR game at the same time, consultants will be made available to each 
team on a first-come-first-serve busis. 



I.S.T. UNITS 

In real life, a researcher may need extra training to enhance his knowledge 
in arras relevant to nis research. Tnerefore, he mi^ht opt to take courses at a 
nearby university. In the FEHR-PRACHCUM game, such "courses" are called 1ST .nts. 
The In-Service Training unit consists of instructional materials which provide 
training in research skills especially relevant to FEHR-PRACTICUM. The particular 
"courses" which are available in any one game are listed below. To obtain an 1ST 
unit, please submit a verbal request to the game manager. 



Number Title of Unit 

I Assessing Success for Complex Objectives 

II Criterion-Referenced or Mastery Testing 

III Computer Format Statements for FEHR Data 

IV Sampling the Subjects to be Stjdied 

V Using the FEHR Secretary 



REFERENCES 

In addition to the topics covered by the FEHR-PRACTICUM I.S.T. Units l;istea 
above, there may be other research skills with which you need assistance. Tne 
following set of programffied instructional lateria'is is recommended as a means by 
which you may achieve a greater degree o*^^ .ompetency in those skills. Tnese 
materials were developed by the Southwes;, Regional Laboratory for Educational 
search and Development, and are available from the American Book Company. 

Educat.onal Criterion Measures by W. James Popha/i' 
(C785fi-000-S) ' 

Sel ecting Variables for Educational Researchr by Paul E. Resta. 
l^oBert I. BakGr 
(C7864-00Q-8) 

Components of the Educational Research Propo sal by Pau^ E. Resta, , 
Robert L. Baker 



RJC 



(C7865-000-3) 

lassifying and 
ill i van 
(C7863-000-2) 



Classifying and Interpreting Educational Research Slu e .es by Howard J 
Sull ivan 
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Simplified Designs for S chool Rese-rch by W. James Popham 
(£7867-000-4) ' 

Choosing an Appropria te Statistical Procedure by Ricnard M. Wolf 
(C7868-000-X) . 

The Use of Library Computer Programs for Statistical Analyses by Richard M. 
WoTf 

{C7869-000-5) 

The Research Report by Paul E. Resta 
- {C7870-000-0) — 



30 
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APPENDIa II 
AUXILIARY REFERENCtS 

1. Blalock, Hubert M., Or. Causal Inference s in Nonexpe rt-mental Research. 

Chapel Hill, North Carolina" OTnversity of North Carolina Press, 196A. 

cial Research, 



2. Blalock, Hubert M. , Jr. and Ann B. Blalock. Methodology in fe 
McGraw-Hill , 1968. ~. ^ 



3. Blank, Stenley S. Descriptive Statistics . New York: Appleton-Century-Crofts , 
1968. 

A. Best, John W. Research in Education (2nd ed.). Englewood Cliffs, New Jersey: 
Prentice-Hall , Inc. , 1970. 

5. Campbell, Donald T. and Julian C. Stanley. Experimental and Quasi -Experimental 
Designs for Research . Chicago: Rand McNally & Company, 1963. 

5. Uressel, Paul L. and Associates. Evaluation in Higher Education . Boston: 
Houghton Mifflin Compan 1961^ ~^ 

7. Edwards, Allen L. Experimental Design in -Psychological Resea rch (3rd ed.). 

Holt, Rinehart and Winston, IncI, 1968. '• 

8. Edwards, Allen L. Statistical Analysis- (3rd ed.). Holt, Rinehart and Winston, 

Inc. , 1969. 

9. Engelhart, Max D. Methods of Educational Research . Chicago: Rand McNally 

and Company, 1972. 

10. Ferguson, George A. Statistical Analysis in Psychology and Education (2nd ed.). 

McGraw-Hill Book Company, 1966. ^ 

11. Games, Paul A. and George R. Klajre. Elementary Statistics—Data Analysis for . 

the Behavioral Sciences . McGraw-Hill, Inc., 1967. 

12. Harman, Harry H. Modern Factor Analysis (2nd ed. rev.). Chicago: Tne 

University of Chicago Press, 1967. 

13. Heermann, Emil F. and Larry A. Braskdmo (editqr-s). Readings in Stoti>_..j C|_rj3r 

the Behavioral Sciences . Englewood Cliffs, New Jersey: Prentice-na i 1 , 
Inc., 1970. 

14. Kearney, Patricia^A.^ Programmed Review of Fundamental Mathematics for nemc. .- 

tary Statistics^ ^ Englewood Cliffs, New Jersey: Prentice-Hall .Inc. , 1970. 

15. Kelly, Francis J., Donald L. Beggs and Keith A. McNeil with Tony Licnelberger 
' " and Judy Lyon. Research Design in the Behavioral $ciences--Multi ple 

Regression Approach . Southern Illinois University Press, 19&57 

16. Kerlinger, Fred N. Foundations of Behavioral Research . Holt, Rinehart and 

Winston, Inc. , 1964. 



\1. Lerner, Daniel (ed.). C ause and Effect (The.Hayden Colloquium on Scientific 
Method and ConceptTT New York: The Free Press, 1965. 



■ - 96 - 

I ERIC 



7 



PLAYERS' INSTRUCTIONS PAGE 13 



18. McFarland, Susan J. and Larl F" Hurefo^ : (ocMtor^J. Man-. tics cinu Mea>ur--- 

ment in the Cla ssroom (i^nd ed.). A Psycnoloo.ca i Foundations of 
Educatio'nlertes . Dubuque, Iowa: Wm. C. Brown '^or^pany Publishers, i7L 

19. Meehan, Eugene J. Uplanation in Sooal Sc ienc e ^-A System Paradigm . Homewood, 

n 1 i noi s : The Dorsey PrPssT H^.^ 

20. Meehan, Eugene J. Value Judgment and Socia l Science — Structures and Proctfoses . 

Homewood, Illinois: The Dorsey Press ,~T55^.' 

21 # Kinium, Edward W. St-atistical Reasoning in P sychology and Education . Joh'i 
Wiley & Sons, Inc. , V"0. 

22. Mouly, George J. Th e Science of Educational Research (2nd ed.). Van Ncstr?nd 

Reinheld Company, 1970, ~~ 

23. Phillips, Deirek L. Knowledge From What? Theories Bno Methods in Social 

Research, . Ct^icagol Pand McNally and Company, 1971. ~~ 

24. Press, S. James. Applied Multivariate Analysis^ Holt, Rinehart and Winston, 

Inc., 1972. 

25. Sax, Gilbert. Empir ic al Foundations o f Educati onal Research , Englewood 

Cliffs, New Je>',jy: Prentice-Hal^l , Inc., lS6S^. 

Smith, G. Milton. A Simpl i fied Guide to Statistics (4th ed.). Holt, Rinehart 
and Winston, Inc . , 1970. 

27. Whitla, O.K. (ed.). Handoook of Measure ment and As sessment in Behavioral 
S ciences . Addi son-Wesley PulDishing Company, 1968. 

• 28. Williams, Frederick. Reasoning with Statistics--Sinplifu^d Examples i n Corr.- 
niunications Research . Holt, Rinehdrt and Winston, i"nc. , 1968. 

29. Winer, B.J. Statis tiCdl Pr inci ples in Exper imental Design (McGr;>w-Hi 1 ] series 

in PsychoTbgyT. McCiraw->ii ll Book Company, Inc., 19b?. 

30. Young,. Robert K. and Donald 'oldman. Intro ductory Statistics .o. Be - 

ha'vioral Sciences (2nd eo. . Holt, Rinehart and'lilTnston , inc.. 
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APPENDIX III 

INSTRUCTIONS FOP OBTAINING INFORMATION FROM 
THE FEHR DATA GENERATOR 



It is useful to begin with a definition of the ten.' us.d in instruct 
especially since many of them have rather snecial meam..qs in the FEHR-PRACTILUM. 
A glossary of terms appears below. 

File Search: A file search refers to a subject's score on a jarticular 
te t (or va Hable) administered some time in the past. Therefore a file 
e c win returi exactly the sa.e score for a ^-Ucu ar ind^v.du 1 each 
time it is used. Since no act j1 testing is ^"volved file sea, ches are 

cheaper than surveys. However, on^y those variables which have YES 
in the FILE column can be obtained a file search. 

.urvey A survey refers to infomation collscied by the administration of 
riilt'or tests at tne .resent Um; that is. at time^ero m the q me^ 
• See Appendices III and I. of your RFP package for ^ ,def;";tion of nme^^^o^) 
Each t^me a survey is conducted t^e tesi.(s) are re-admimstered 1"^!;^°;^ 
w? 1 ge^s^ewhat different scores each time because ^Vn'^n^n^nd x IV of Tour' 
The costs of a survey score on one .erson is tabulated in Aooend^x IV of your 



Treatment: A treatment in FEHR refers to the nrogram which 

^Tfr^kr treatment. T^sts can be administered '^^f ^^.^^^^e er^! 

-^-^ra^:nit?f%?:ri:^ ^^we^e^: ^.^^^^ — 

available are all specified in Appendices III and JV of your RF^ 



his 



would be adva*>tageous in cases^ where it co^ts more to adm nister 
to four small, groups than to a single larg#^^ne. 

" card: ^ card refers - a sin,l, '^Vj'^^'rtriairSsIaUrl- 

mller of characters-includinq letters, numbers. ^,'^1^^^°' ' ^^^.'irder 
"ny one card is 80. Cards must be entered to .a computer in a soecific oraer 
which is prescribed by the instructions whir', follow. ^ 

Line- A line refers to a nosition on a nrivate comnuter fMe. J*;^^^^" 
' TH5^qht ol as successive lines on a printod page. A line ^" ^Jl^^^i;/" 

exact counterpart of a card. -Again, the iinei m.st aooear in the order 
v.- specified by the inst uctions. 

Column: A cclumn, refers to the position from l€ft tn right on \l'l^ ^'^ 
TT^T- The first position is, column 1. the second column 2. and so on. 

Columns are exactly equivalent to spaces on a tyoew-iter Each column 
S contarn any legal typewriter character-including a blank. .Not. that a 
^ blank is entered with a typewriter space bar. 
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GAME MANAGER'S NOTfS FOR PLAYERS' INSTRUCTIONS: PAGE 15 



Step 1 Every request to the FEhR Data Generator must begin with a key card. 
The first seven columns (spaces) of the key card contain a series of 
switches or key: which control tne operation of a anety of optional 
features built i..to the FEHR Data Generator. These keys are not des- 
cribed in the Player's Instructions because their operdtio^ should be 
under direct control of the Game Manager. We recommend that the card 
or cards for step 1 ue supplied to players by the Game Manager. Fill 
" out the first seven columns o'f the key card using the table below. 

STEP 1. CARD 1 , ^ , , ~ 

toimn key Operation of key (Note. A plank can be subst ituted for zero.) 

• 1 0 No sumnary statistics are printed. . . - u 

1 Means and SD's for all variables are printed ai the end of each 
new file search, survey» or treatment. 
' 2 Means and Sli's for all variables are printed at the end of ear i 

sub-unit within a file search or survey. N ot used for treatmtnts. 

2 i5 Only the regular printer output (device b) is obtainea. (Tnese 
are not suitable for direct analysis because they contain many 
titles ahd the like.) \ 
1 Data is output on computer file device 7 as well as on tne regular 
printer (device 6). Your computer center can explain how to punch 

the device 7 results on cards or store t hem on disk. 

— -3 g Variable headings are prl^ited for each new s>ib-unit within a file 

search or survey, but (as above) only once for each new treatment. 
1 Variable headings are printed only at the beginning of each new 

file search. ^ — ^ — . ■ j- 

7 0 The costs are accumulated and continuously compared with the inaxl- 

mum budget entered by the team. When the charges exceed the bud- 
get by S%, the request is aborted. u ^ * • 
1 Cost are computed as above, but there is no abort If the budget is 

— 5 013 — This IS the number of subject IB's to entered on each Hne. 

Normally a key of 0 (zero) is used to inaicate ten lU's per line. 
However, you would use a 1 here if the card outputs from a previous 
survey or file search- were to be used to identify Jthft„subjects in 
a subsequent run. Any other number between 2 and 9 c^n be used if 
the local devices require it. but note that blank ID are ignored. 

— 5 5 Variable -cores will L»e returned for each legal . ID entereo . 

1 Some students will orop out or move within each survey or experi- 
mental treatment (but not in a file search). Different students 
will drop out for each different computer run, but the proportions 
will be about the same. These proportions (probabilities of at- 
trition do vary from problem to problem, however,and from group 
to group within problems. , . 

— 7 0 The usual built-in treatment effects are in operation. 

1 The built-in treatment effects are each multiplied by a signed 
decimal constant. This allows the Game Manager to magnify or 
decrease differences among treatments. It would normally be used 
only when there were Urong pedagogical reasons for changing tne 
treatment effects. If a 1 is entered in column 7, a treatment 

multiplier card must follow immediately aft er the key card. 



' Thi" treatment multiplier caro is necessary onTy if a 1 appears in column 

7 or the key card. It contains a multiplier for treatment 1 in columns 
1-4, for treatment 2 in columns 5-8, for treatment 3 in columns 9-12, and 
so on up ta the total number of treatrr.ents. Each multiplier should be 
a decimal number. 
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PLAYERS' I-NSTRUCTIONS PAGE 15 
INSTRUCTIONS FOR PREPARING CARDS FOR THE FEhR DATA GENERATOR 
NOTE: Instructions preceded by an asterisk(*) are ODtional. 

NOTE: Carefully read the mOTFS at the end of each stop before beqinmnq the sSep!. 

NOTE : If you are at a terminal read Mine' for 'card' in the following instructions.- 

NOTE : Each step consists of one or more cards. Check to see that yc. have put infor- 
mation in tne correct columns. , 

NOTE^; Ask your Ti^me Manager' for the input and output maximums to be used with your 
computer sys;:em and enter them below: 

. The maximum number of subject ID's to be entered on one line or 
card is . 

. In any file search, survey or treatment, the maximum number of scores 
per subject which can be printed out is . If you need more 

than the maximum number of variables, enter MO'Rt at Step 8 (for file 
searches and surveys) or Step 13 (for treatments), and complete a new 
request using the same ID's. 



STEP COLUMN 

NO. NO. ' . DESCRIPTION OF PARAMETER ENTERED 



Xl) See Game Manager for the initial c^ird or cards. 



(2) 1 Enter 1 for file search, 2 for suiTvey, 3 for treatment. 

(2-11) Enter player-unique number (e.g., social security) for 

random number generator. 
(12-72) *Enter player or team name, date, etc. (This will serve 

to ''dentify your output). 



NOTE' Steps (3) to (8) un paqes 14 and IS are for filo sea-che^ and surveys only, 

' while steos (9) to (13)' on pd<]es 16 and 17 are for treatments only. If you 

are^doing a TREATMENlT, skin nanes 14 and IS and no directly to t^o (P) , oaqe 



16. 
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NOTE : All cards on tnti next two pages are for file searLn ^nd survey only. 
NOTE : Go to page 16 if you are running a treatment. 



STEP COLUMN ' 

NO. NO. DESCRIPTION OF PARAMETER ENTERED 



(3) (1-10) Enter budget maximum allocated for this file search or 

survey. (Use'a decimal point. Do not use a S. Do not 
use any conwas.) > 

(11-71) *Enter team's verb&l title for this run. (This wiVi serve 

to remind you o'f the nature of th^s run.) 

NOTE : Make su;3 the verbal title does not begin before column 11 It can be any- 
where in columns 11-71. 



(4) (1-3) Index of the first variable to be mea:- jred (right justified, 

e.g., 003). 

4 Blank 

i5-7) Index of the second variable to be measured (" 

8 Blank 

(9-11) Index of the third variable to be measured (" 

12 ' Blank 

(13-15) Index of the fourth variable to be measured ( 

/ Alternate blanks and three-digit indices. 

(77-79) Index of 20th variable to be measured. 

NOTt 1 : You need not enter indices for 20 variables if you reat?^' want less. 

NOT^ 2 : Each variable index must appear on the card exactly as it appears in the 
catalog of variables. 

NOTE 3 ; ^ Your game marj&gev^ can tell you if your output deviee limits you to fewer 
. than 20 variabl. ^. 

V- ^ U- 



'^(i) j ,(1) Punch 0 (zero,) here unless you know kow to use the 

— ^ / secretary. If you do Icnow how/ then" enter thf? appropriate 

cards. Your game manager can help you with thi4^ convenient 
' ' feature. . 

• — — _ 

(6) ^(1-2) . Enter the number of cards of subject ID's that wijl 

follow. The game mana'ger can tell you how many fo's you 
^ can enter_on one card. See notes for step ,7)lbefore 

» proceeding. ^ 
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STEP COLUMN 

NO. NO. DESCRIPTION OF PARAMETER ENTERED 

(>') (1-7) Enter the first ID (e.g., 0102003 is the third student in 

the second sub-unit of the first unit). 
8 Blank 
(9-15) Enter the second ID. 

16 Blank 

.... Continue with as many ID'S as can fit on one card. Each 

ID should be followed by one blank. 

NOTE 1 : Anytime an entire sub-unit is to be in the sample, then use 000 for the 
last 3 digits, e.g., 0110000 indicates that all individuals from unit 1, 
sub-unit lO, are wanted. This will count as one ID. 

NOTE 2 : Do Not use -(dashes) in specifying the ID'S. 

NOTE 3 : Continue doing this for a: many cards as were indicated in steo (6). 

NOTE 4 : To keep simulated costs down, you should complete samoling all sub-units 
in one unit before moving on to another unit. 

NOTE 5 : To keep simulated costs down, you should complete sampling all ID'S in one 
sub-unit before moving on to another sub-unit. 

NOTE 6 : Assume that you can enter 10 ID'S per .card unless the game manager says 
otherwise. 

(8) (1-4) Enter END if you have completed the run. 

MORE if you wish to add variables to the present 

study by returning to step (4) 
NEW if you wish to begin another study at 

step (2) 

NOTE: The first character must be Dunched in column 1 . 
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NOTE :^ All cards below are needed only for tffte treatment routine. 
NOTE: Fill out one set of Cards for stens (9)-(13) for each treatment group you wish 
^ to define. - 



COLUMN 

NO. DESCRIPTION OF PARAMETER ENTERED 



(9) (1-2) Enter the number of cards of subject ID'S in this treatment 

group (right justified). (The game manager can tell you 
how many ID's you can enter on one card. See notes for step 
(7) before proceeding.) No more than 99 cards of ID'S can 
be entered. 

3 Blank . , 

(4)' Enter the number of measuring times (Pight justified) (e.g.* 

pre and post tests require 2 measuring times; a post test 
alone requires 1 measuring time.) No more than 9 measuring 
times can be entered. 
5 Blank 

(6) Entev the number of distinct treatments given to this group 

(right justified). (Usually 1 unless a series of treatments 
is administered.) No more than 9 distinct treatments can be 
entered. 

7 Blank 

(8-17) Enter the maximum budget allocated for this treatment group. 

(Use a decimal point. Do not use a $. Do not use any 
comnas.) 

(18-77) *Enter team's verbal description of tifis treatment group. 

(This will serve to remind you of th'e nature of this treat- 
^ ment group. Be sure you do not start this before column 18.) 



(10) (1-7) Enter the first seven digit ID of a subject or sub-unit (see 

notes for step (7), page 3). 
8 Blank 
(9-15) Enter second ID. 

T6 , Blank 
(17-23) Next ID 

24 Blank 

Continue alternating ID's and blanks. 

(73-79) Ente"" ^^nth ID. 

80 Blank 

NOTE 1: Continue entering subject ID's on cards until all subjects are entered. 
NOTE 2 : The number of cards of subject ID's must agree with the number in columns 
(1-2) of step (9). 



(11 1) (1-2) Enter the time at which the first measurements are taken 

(right justified) (e.g., 01 is at time 1). 
3 Blank . ^. . 

(4-6) Enter the index of the first variable wanted at this time (use 

all three columns) 

7 Blank " ^. / 

(8-10) Enter the index o* the second variable wanted at this time luse 

all three columns). . . 

Continue alternating blanks and variable indices (maximum » 19 

I indices) 

f NOTtl: Your game manager can tell you if you are limited to fewer than 19 variables. 

TOTTT: All entries are right justified. 

NOTg 3 ; Any pretest, of necessity, is given at time 00. 

"^^^'^^ See Catalog of Treatments for appropriate time units. 

ERTC^^ See Catalog of Variables for appropriate variable indices. . 
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STEP COLUMN 

NO. NO. , DESCRIPTION OF PARAMETER ENTERED 



(11.2) (1-2) Enter the time at which the second set of measurements is 

given (right justified). 
3 Blank 

(4-6) Index of first variable wanted at second measuring time. 

7 Blank , 

(8-10 j Index of second variable wanted at second measuring time. 



(etc 

) ste. 
(4 ) of step 



NOTE ; Do step (11.2) only if you asked for more than 1 measuring time In column 
" ' " (9). . 



(11.3) Complete cards as above for each measurement time. 

NOTE: The number of cards for step (11) must agree with the number entered in 
columns (4 ) of step (9). 

(12.1) (1-2) Enter beginning time for the first treatment v'this is 

always 00). 
3 Blank 

(4-5) Enter Index of the "first treatment (e.g., 01). 

NOTE ; See Catalog of Treatments for appropriate treatment indices. 



(12.2) (1-2) Beainning time for the next treatment for this group. 

(When the second treatment f:r this group begins, the 
first treatment will, automatically end.) 



3 Blank 
(4-5) Index of second treatment. 

NOTE ; Do step (12.2) only if you asked for more than 1 distinct treatment in column 
(6) o f step (»). - 

(12.3) Repeat card above until all treatment changes have been 

: defined for this treatment grdup. 

NOTE : The number of cards for step (12) must agree with the number entered in 
column (6) c step (9). 



n3) (1-4) Enter END if you have completed the run. 

MORE If you wish to specify another treatment group. 

If MORE,, then you will repeat steps (9) to (13) 

for the new group. 
NEW if you wish to begin again at step (2). 



no- 



es 



SECTION III: ^ . 

IMPLEMENTING THE STANDARD REMAR PROBLEM * ^ ' 

The first task in iniplementing pEHR at a new site is to operational ize 
the computer program for the standard REMAR problem. For -all first-time 
orders, we sli p an intact deck of punch cards containing the FORTRAN source 
code for both the main Data Generator and the REMAR problem packet. Accompany- 
ing the card deck there will be a list of instructions regarding the adaptations 
(if any) which must be made to the program to accommodate it to the FORTRAN IV 
compiler in your specific computer system. Although these are few in number and 
generally prescribe minor changes, they are site-unique and therefore cannot be 
anticipated here.. It is nighly desirable to obtain the services of a systems, 
prograiiTOer at your computer center during the first compilation of the program 
and the setting up of efficient program storage and use procedures. Efficiency 
Is particularly important if you plan to add more FEHR problems at a later date.' 

Validating the Program . Once the program has been compiled, it should 
be rea<l>' to operate. Nevertheless, it is wise to test its ooerability by using 
input whose effects are precisely predictable. Consequently, we supply a number 
of sample requests to the Data Generator, a display of each output, and a thorough 
discussion of the key characteristics of each. However, in the context of this 
manual, it is both convenient and more efficient to combine program validation 
with the larger task of familiarizing the users (i.e. potential game managers, 
research consultants, and instructors of courses to be asst)ciated with the prac- 
ticum) with the capacities of the FtHR Data Generator. In aiscussing the fa.tiiliar- 
Ization requests appearing later in this section, we shall point out the output 
characteristics which indicate program malfunction. 

In the remaining pages of this section we provide specific step-by-step 
guidelines for planning, organizing and running a practicum session. However, 
in many instances prescriptive instructions are impossible because the system 
was designed to adapt to aifferent user purposes. In these cases the guidelines^ 
are necessarily more general. However, additional guidance is provided by three 
contracting illustrative examples of specific uses the REi^R problem which follow 
immediately after the general guidelines. 

The practicum session serves fundamentally different purposes in the three 
examples. In tne workshop example the practicum is a self-contained workshop 
intended to upgraoe the program-evaluation skills of new researchers at an 
educational research ana oevelopinent laooratory. In tue module example. 



on the other hand, the practicum serves as the core experience for a 
oroblem-based training module designed to provide the basic research com- 
petency p'-erequisite to the Ph.D. degree in Education, finally, in the 
laboratory example, the practicum is used to provide "realistic" laboratory 
exercises which require the application of specific skills from a course in 
analysis of variance techniques. 

* The remainder of this section is organized under five main headings. 
In order of presentation, these are: 

I. General Guidelines (pages to ) 
II. The Workshop Example (pages to ) 
' III. The Module Example (pages to ) . * 

IV. The Laboratory Example (pages to ) 
V. Some Suggested Extensions (pages to ) 

* 

GENERAL GUIDELINES 

\ 

The guidelines presented in this section are intended for the novice 
user who has not had any previous experience wiih FEHR- PRACTICUM. However, 
exactly the same procedures should be used^ whenever a new problem is intro- 
duced, although with considerably less 'emphasis on activities designed to 
familiarize one with the computer program £er se_. 

There are ten , steps in planning a FEHR- PRACTICUM session. Although 
the emphasis placed on ?.ny one step may vary with repeated uses of the 
practicum, it is essential that each step receive careful attention for each 
new session. In chronological order, the ten steps are: 

1. Defining the goals of the proposed session. 

2. Choosing the personnel to conauct the session. 

3. Familiarizing personnel with the Data Generator. 

4. Familiarizing personnel with the practicum process. 

5. Choosing a place to meet. 

6. Selecting materials for the session. 

7. Choosing the practicum tasks to be completed. 

8. Planning the instructional activities for the session. 

9. Evaluating the practicum products of the session. 

10* Evaluating the instructional ef ectiveness of the practicum sess^or 
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!• Defining Session Goals ^ ^ 

The prospective FEHr/pRACTICUM user will have begun this manual with 
some broad instructional pijrposes ii; mind.. Having read this far in the manual, 
he or she should have a general idea of what the practicum can (and cannot) 
do for them. To facilitate planning, it is useful at this point to specify 
the session goals. That is, precisely what will the. player- trainees be ex- • 
pected to learn by participating in the practicum. These need not be stated 
as skill-by-skill behavioral objectives. Goals involving broad constellations 
of skills are more useful for instructional planning. The degree of speci- 
ficity required is illustrated by the examples appearing later in this section. 

2. Choosing Personnel for the Session 

In previous sectidns **e have mentioned two staff positions associated 
with a FEHR-PRACTICUM session: ' the game manager and the consultant (or con- 
sultants). In these discussions, we have attributed to the game manager the 
functions of both-i nstructional planner and supervisor of the practicum 
ac:^Tvities. However, in many Institutional settings it will be more con- 
venient to have these two functions performed by different people. For that 
reason, we have identified three positions or functions below and listed the 
requisite skills for each. Unfortunately, the amount of time required in 
each function varies so widely from session to session (depending on purposes) 
that even broad guioelines cannot be established. For guidance, see the time 
allocations in the three examples. 

Instructi on . The instructional function Is usually perfo^ed by one 
person. He or she must have the following characteristics: 

a. Extensive training and experience in educational research. 

b. Some experience in teaching research methods. 

c. An If.timate knowledge of the practicum process and experience 
with tne problem being used. 

d. Considerable experience in using all available statistical 
programs. 

Game Manager . In the strictest sense, the game manager Is required 
only to' coordinate activities, run meetings, distribute materials, facilitate 
interaction with the computer, and act as the bank clearinghouse for all 
monetary transactions. The last two can be time consuming activities if all 
computer runs are batch-processed once a day and all checks recorded by hand. 
See the examples for illustrative time allocations. 
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In any case, the game manager should possess the following charac- 
teristics*. , * ' 

a. Competent in the use of canned programs at the local computer 
center. 

b. Experienced in doing each practicum task with the particular pro- 
blem RFP being used. 

c Reput-^tion for promptness and dependab41ity. 
d'.~C?.pable of keeping- neat, accurate, and comprehensive records, 
e! Perceived as friendly and approachable by the player- trainees, 
f' Good general knowledge of research/evaluation methodology is 
' useful but not absolutely essential. 

Consultants . Again, the number of consultants or the amount of time 
each one should be available depends on the instructional arrangements for 
the session-s^e the examples later in this section for guidance. In all 
cases, however, each consultant should possess the following characteristics: 

a Competent in basic statistics. 

b" Competent to run the (relevant) canned statistical routines 
* available on the local computer. 
- c Considerably more training and experience in educational research/ 
evaluation than any of the player- trainees. 

d. A good yrasp of the basic FEHR principle., and some experience with 
each of the tasks involved in the particular problem being used. 

e. Free to meet at times convenient for the player- trainees, 
f*. Perceived as friendly and approachable by player- trainees. 

3. Familiarizinq> Personnel with the Da ta Generator 

The purpose of tnis section is to familiarize the potential users (game 
manager, consultaht. or instructor) with the operation of the pata Generator. 
This is done by providing a series of sample requests designed to demonstrate 
various features of the computer program. Specific examples must necessarily 
come from^a particular problem. For reasons given in section II. we have 
chosen to base all illustrations on REMAR. However, users having several 
operational problems may wish to develop similar computer-familiarization 
examples based on other problem areas. It is important to note that, while 
some substantive ififomation may be derived from the results, examples such 
as these are not intended to provide information which is necessary-or even 
useful-in "solving" the problem concerned. 
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The term "request" as used here is defined as a set of ^uach ^rds 
^ which relate to a single research operation. That is, a file search, a sur- 
vey, or an "experiment" involviTng the administration of different educational 
treatments. It is possit>4e to include several request^ in the s^ computer , 
run. For example, a single computer run might contain file searches f rem ^ > 
several team's, a couple of surveys, and one or more "experiments which ad- 
minister two or more treatments each. In general, it is a more efficient 
u$e of computer time (and therefore cheaper) to include as many requests as 
possible J n each computer run. _ 

These are, however, at least two cases which require separate computer 
runs. The most common of' these is sequential dependence among requests. For 
example, teams frequently must run a survey to determine which subjects they 
>i1sh to put in an experiment • The second case occurs when the G4me Manager 
wishes to use a different set of optional features (defined on c^rd 1): one 
must begin a new computer run whenever a change in options is desired. 

One arrangement which provides optimal computer efficiency is for re- 
quests from all teams to be collected by the Game Manager, sorted into com- 
patable runs and then Vun in batches on some regular schedule. 9ut even if 
this arrangement is used, it is both more convenient and pedagoglcally 
#sireable for each team to be required to punch the cards for their own re- 
quests. However, this pattern would normally be used only In institutions 
"where computer time is at a premium. 

In most installations, it will be grossly inconvenient for the Game 
Manager to collect and run the requests personally. |n this case we recom- 
mend that the teams be taught to run their own requests. This was the usual 
pattern used in the FEHR field evaluation studies. It was our experience 
that, with minor supervision for efficient request runs, the additional ex- 
pense of running programs team-by- team was minimal --on the order of a 10% 
Increase. 

We shall present examples of five separate computer runs containing 
a total of seven requests.- Computer listings of the card dec1( ^e suppTted 
for each computer run. For each request, we shall provide verbal descriptions 
''of the information desired, the computer options^ illustrated, and comments 
about the function of^ various elements in the body of the request. Becajse 
the comments continually refer the reader to the instructions for preparing . 
a request for the Data Generator, we recommend that pages^to-\)e (temporarily) 
* f 



nwoved from the iMnual for ready reference. The first request will be dis- 
cussed on a line-by-line basis; after that, only the unique aspects will be 
discussed. For the reader's convenience, each computer listing and the 
corresponding descriptive material have been placed on facing pages wherever 
jtossible. Yo accommodate longer explanations, single spacing will be used 
for all coamentary materials. < ^ ' 

Computer Run #1 .. (See the card listing on the page opposite) 

There is only one request included in this run: a demoi,^ ration of a ^ 
file search. The resulting computer printout! appear on the pages following 
the listing pf the requiest. 

Request 1 . The information requested consist of the scores of the 
last person from each class in the seven schools on the following variables 
(in order): race, sex, "'father's education, Stanford Binet IQ, criterion, 
referenced tests of addition, subtraction, multiplication, and division, 
and Stanford Achievement Test sub-scale scores for airthmetic computation, 
arithmetic concepts, and mathematics application. These particular variables 
were chosen because they facilitate validation of the. program. Similarly, 
the Tasi person in each class Is used to check the pbpulration parameters. 

O ptions . The options MS'ed for this run are defined on carJ 1. Because 
a 1 appears in column 1, means and standard deviations will be printed at the 
end of the file search. The second 1 causes the computer to punch (or print) 
the data on file device 7, and the third 1 causes the computer to print only 
one headins instead of separate headings for each. The remaining zeroes on 
the first card result in a budget abort if the actual cost exceeds the esti- 
mated cost by 5X, a default value of ten subjects p«r line, no attrition 
feffective only for treatments), and the usual trea^jnent effects. 

Comments 

(1) The first number on Card 2 (col 1) calls a file search. 

(?) The number 0123456789 in columns (2-11) is usually replaced with 

a user's social security number (or student number). Changing 
c this number do^s not affect a file search, but will change thfi 
obtained scored from a survey or treatment. 

(3) Card 3 establishes a maximum (FERH money) cost for this request. 
If this amount is exceeded by 5St, the computer program will abort. 
The. user must, then recalculate the budget. The cdst of the pro- 
posed file search is calculated by sunming the following: 

Cos* -f inUiating af^'search $ 10.00 

Cost of entering seven schools (7x20) 140.00 
Cost of enitering sixty-nine class.ooms 

(69x0) 000.00 
Cost of 11 scores each on 69 people 

(6^x11)^.05) . 37.45 

Total $187 ..45 
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CARn CONTENTS OF CARD (BEGINNINH IN WIMM 1 ) 



1 
2 
3 
U 
5 
6 
7 
8 
9 
10 
11 
12 
13 
lU 
15 
IC 
17 



V- 



ioi73uSG78q RFnursT #1. BiinnFTpn file sfarch. 
nniinnn; in's arf last" person 'NFACh class. 

002 003 OIG 033 075 07n 077 07R 0U2 0U3 OUU 
n • ■ • X 

JJoio28 010202f. 0103027 010U029 0105028 010Ca29 0107029 0108028 0109028 0110029, 
nm n??r 0111029 0112001 lOROOl 0000000 - " , 

^ ^ =^ 0308028 0309027 .3100.S 

SroiS^r, Vulnll lllllll 0U0-U02.. 0.05027 OUOr.026 0UO7025 0.O8.27 0Ud9027 0U10026 

•S5J1"0?C SsJzSf;. 0503028 05nU025 0505027 0506027 050702G 050802,8 0509027 

0001028 0f,0?O28 0R03029 OnOU028 0R0502J JR0n028 0P07050 J^J-^O" , 

0701029 0702029 070302'? 070U030 0705031 0700^29 0707030 0708028 0709029^ 
END " . ' • 
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Computer Run #2 



The second computer run contains requests number 2 and 3. 

Request 2 . This is identical to Request 1 excefit for columns 1-10 
of card 3, the maximum cost. This amount has been reduced to $10&— a figure 
below the correct budget given in Request 1. The results of the aborted 
request are displayed on the following page. 



(1) Note also that card 17 new reads "NEW" instead of "END." The 
effect of this change is to send the computer back up to step 
2 of the instructions to fi.„i out what additional information 
is needed. 

(2) The abortion of Request 2 does not affect Request 3. 

(3) Your data oiJtput should be identical to that from Request 1. 
up to the point of abortion. ^ 

Reques^t 3 . Tire information desired in this request fs the same as 
that desired for Requests 1 and 2, except for two things: (1) we are now 
usirg a survey, *and (2) we now desire summary statistics for each school. 
For this reason we haVe separated the ID's by school. 'To conserve space, 
only the first three schools are used in the example. Technically, we now 
have three independent surveys. However, we have labelled them as sub-sets 
of one request because they are after the same information and use the 
same sub-routine (furvey) to get it. 

Options . Since this is on., the sam^ computer run a: Request 2, it 
must use the same options. Card 1 for this run is identiCb'l with that in 
the first computer run, as described above. 



(1) The test scores returned in this survey will differ slightly 
from those returned in'a file search. Moreover, the scores for 
one individual will change from survey to survey due to (simu- 
lated) testing error. 

(2) It is possible to obtain the identical scores from a survey by 
using exactly the same set of cards. Specifically, the number 
entered in columns 2-11 of card 2 must not be changed. In 
addition, the order of cards in the entire run must remain the 
same. For example, reversing the order of Requests 2 and 3 in 
this run would produce different test scores from Request 3. 
It would not change the results of Request 2, since a file 
always contains the same score regardless of when it is accessed. 
Note that file scores- contain measurement error too— it Just 
returns the same error every time. 

(3) The cost of each test score in a survey differs from test to 
test, and fs considerably more expensive than the same score 
obtained from a file. For example, the Stanford Binet score 

. costs $12.65 per student in a survey (the cost of the test oius 
the cost of an expert spending 1 1/2 hours to give the test). 



Comments. 



Comments 
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CARD CONTENTS OF CARH (BEGINNING IN rOI.UMK 1 ) 

1 iiinnno 



2 iri23!l5678<) RFOUFST #2/FILE fiFARCM. SAME AS HRST BDT BUPPET TOO SMALL. 

3 lon.nnnnnn in' s, are last persom in each plass. 

l» . . 002 003 OIC 033.075 076 077 078 0I»2 0i»3 Oi»i» 



5 0 

6 10 



7 0101028 0102026 0103027 0101*029 0105028 01CG029 0107029 0108028 0109028 0110029 

8 0111028 0111029 0112001 108001 0000000 ' 

0 0701050 0202028 0203028 0201*027 0205029 020G03U "0207031 at^aa^c 

5 S30IS27 0302026 0503025 050U027 0505026 050G027 0507027 0508028 0309027 0510026 

VX. lllia^ll !!'J3S27'0U0..O2«* 0U05027 01*060 26 OUO 70 25 0U08027 0U09027 0U10026. 



10 
11 

12 _ 

\l 0501076 0507027 0503028 0«;ni*025 0505027 0506027 0507026 0508028 0509P27 

0601028 0602028 0603029 060102^ 0605029-0^06028 0CT7050 0608029 

ll nilllll 0?n2029 0$05029 070l050 0705051 0706029 O7O7050 . 0708028 0709029 

_ 18 201751*56739 REOUFST #5. SURVEY BY SPHOOLS. 7 

5 19 187.500000 SCHOOL l-IO'S ARE LAST PFRSOM IM EACH -M«S. 



20 

21 0 

22 02 



002*003 016 055 075 076 077 07ff 01*2 01*5 Ol*U 



25 S1OIO28 0102026 0105027 0101*029 0105078 0100029 0107029 0108028 0109028 0110029 

21*- 0111028 0111029 0112001 108001 0000000 
25 MEU ^ 

7r. 20173U56789 REOUEST *3 CONTlNUFn. 

27 130.10000O SCHOOL 2-in'S ARE LAST . PERSON If EACH CLASS. 

002 003 016 033 075 076 077 078 01*2 0li5 Oi*i* 



28 

29 "0 
50 01 



51 0201050 0202028 0205028 020U027 0205029 020605«* 0207051 

52 NEW 

7017^U56789 RFOUFST #5 CONTINUEO. 

It Hi- srssc, ii^^ ^ii i^vsr, ^fh. «„■ on o« o» ... 

36 0 " 

\l S3OIO27 0302026 0505025 030«*027 0305026 05P6027 0507027 0308028 0509027 0310026 

39 r*311026 03120^7 0513026 _ . 

uo .:Nn 11 

L .' : . 
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but It Would cost only S cents to retrieve it f rom ja file 
♦f It had been administered previously--in this case it had not. 
Using the costs apoeartng in the catalog of variables (pages 
15-24 of the REMAR RFP oocumenj)^ we discover that the cost 
of obtaining scores Oil alT Tl' variables from one subject is 
$14.30. (S«e table) 

•Therefore, the cost of each school survey would be $10 Initiation + $20 
school overhead + $14.30 times the number of students surveyed. The 
estimated costs on cards 19, 27, and 34 were computed in this way. 



Variable 


Race 


Sex 


Path 
EOUC 


S ^ 
I Q 


CRT 

m 


CRT 
SUB 


CRT 
MULT 


CRT' 
DIV 


SAT 
COMP 


SAT 
CONC 


APPL 


Index 


002 


003 


016 


033 


075 


076 


077 


078 


042 


043 


044 


COST 


.10 


.10 


.35 


12.65 


.05 


.05 


.05 


.05 


.30 


.30 


.30 



NOTE: The output for Reguest 2 Is opposite. The output for rec^uest 3 
begins on page 
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Com puter Run #3 

The third "compute run contains two requests: a survey of nine Intact 

<4asses, and a-one--.g'^oup experiajetit. - - 

Request 4 . In this survey the same set of variables are used as in pre- 
vious surveys, but now we are locking at all the subjects In nine entire 
classes, and using the FEHR secretary to select the cases to be printed. . 
Options . There are three changes on card 1. The 2 In column 1 results 
In summary statistics for each class;. the zero in column 2 si/ppresses 
punching on output device 7; and the 1 in column 4 deactivates the bud- 
get abort, making It unnecessary to specify a budget on card 2. 
Comments . There, are two unique features in this request; the use of 
the rtHR secretary and the use of Intact groups. 

(1) Any non-zero nni)jer In column 1 of ca'^i 4 activates the secretary. 
See 3ST Unit V in Appendix of this manual" for. directions on using 
the secretary. We have arbitrarily chosen to select from the nine 
Intact classes listed later only those subjects who have IQ's 
greater than 85 and a score of zero on one or more of the CRT .tests. 
Cards 4-8, inclusively. Instruct the secretary to select these 
subjects. 

Not e that you are charged for testing all subjects In each class 
even though they are not all printed. (The tests must be given to 
determine their score.) 

(2) The id's entered on card 10 all end in three zeroes. This is a signal 
to the computer to retrieve all of the subjects from the school and 
class Identlfierf by the first four dibits.- (See note 1 of step 7.) 

Request 5 . Th<s is the first request to the treatment subroutine (see 
column 1 of card 12). We have chosen to use one intact classroom, and to 
administer treatment 1 for the first 4 weeks; treatment 2 for weeks 5,6,7 and 
8; treatment 3 for weeks 0,10,11 and 12; and treatment 4 for weeks 13,14 and 
15. (Tiiere is not a 16th week available in this problem— see Appendix III, 
page 14 of the REMAR RFP document.) To access the effects of treatments, the 
.entire test battery was administered at the beginning of the experiment and 
again after survey treatment. This pattern was chosen solely to demonstrate 
the possibility of a series of treatments and not because it makes educational 
sense; 
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'«R;rrorr«*"s;rn;:To_THt ^E^^?,f|^„^^^^sr«!;?i,.'"•;oT^.^^^S"By''^E"cA'R^s below. 



ENTER THE SYSTEM CARDS TO Rllf! THE RE^IAR PROPLEH. 
CARP norJTENTS OF CARD (nERINMINR IM POLUMf' 1 ) 



1 

2 

3. 
k 

6 
7 
8 
9 
10 
11 
12 
15 
Id 
15 
IG 
17 
18 
19 
20 
21 
22 
25 
2li 



2 OU r,T 085 05 EQ 000 
2 01* GT 085 on EO 000 
2 OU RT 085 07 EQ 000 
2 OU GT 085 08 EO 000 

SioiOOO 0505000 05r.9000 0601000 0605000 0608000 0701000 0705000 0709000 
0505000 0700000 

00 075 07G 077 078 0U2 0U3 OUU 002 003 016 033 
OU 075 076 077 073 0U2 0U3 OUU 
08 07'; 076 077 078 0U2 0U3 OUU 
12v^75 076 077 078 0U2 0U3 OUU 
15 075 076 077 078 nU2 0U3 OUU 
00 1)1 / • 
OU 02 
08 03 
. 12 OU 
ENO 
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Options. Ihe options chosen at the beginning of this run are still 

in force. 

Cownents . 

"(1) Notice that there Is no option to use the FEHR secretary during a 
treatment. ' 

(2) The independent (moderator) variables (race. sex. father's education, 
and IQ) are measured only at time 00 (pretest). It is wasteful to 
measure them again since (except for IQ) they cannot change. It is 
however unnecessary to measure these variables even at pre-test time 
If a survey has been done prior previous ly-slmply select the appro- 
priate punchcards from the survey and Interleave. them with the data 
from this experiment. 

(3) Note that card 13 defines t^e number of ID cards in columns 1-2, 
the niinber of measurement times (5) in Column 4. the number of dlf- 

" ferent treatments (4) in column 6. Consequently, the ID cards must 
be followed by five measurement cards and four treatment cards. 

(4) Note that the variables measured at pretest time (card 15) are the 
• sane as those used in the file searches and surveys, above, but 

that they are listed in a.dlfferent order. The Independent variables 
are listed last so that the dependent variables coninon to all the 
measurement times will always appear in the same position on the out- 
put cards. This trick greatly simplifies computer analysis of 
repeated-measures data. 
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rABn*: por a sunns'iiON rn thf fehr tata Rrfn-RATOR. computer rvv ^'i'^'RFp 

lujTn ^MfsYSTei CARHS TO RUN THF RFMAR PROPI.FV. FOL|.nWFn BY THF CARPS BELOW. 
CARP CONTENTS OF CARP (BFRIMNINfi IN COUIHN 1 ) 

_ i 

1 inuQiii -- ■ 



k 



5 0505onn 070Qrno 

G no 075 OTR 077 078 0l»2 01^3 ni»l» 002 003 MB 033 

7 0l»;075 070 077 078 0l»2 01^3 OI»l» 

8 or. 075 070 077 078 01*2 0I»3 OI»I» 

9 * • li 075 070 077 078 0l»2 01^3 OLk . 

10 15 075 07C 077 078 01*2 01^3 Ol»n 

11 p;o 01 

12 {II* 02 . . * 

13 08 03 

lU "12 Ok ' , • 

15 . 'END 
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Computer Run #4 ; Request 6 ^ 

The sole purpose of computer run Is to dennmstrate the modification 
of treatments by a Game Manager. This option Is Involved by placing a 1„ 
In column 7 of the first card. When this Is done, a treatment card must 
follow Iwnedlately. Card 2 tells the computer* to multiply every treat- 
ment effect by JO. In all other respects, request #6 is identical to re- 
quest #5. The results of this treatment modification can be seen by com- 
paring the outputs of request 5 and 6. 
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^ Computer Run IS. ^ ' 

#^The fifth computer run contains only one request .^Nlt^^er, that request 
Involves th«» administration of four different treatments to the same Intact 
class. " . ' 

Request 7. The use of the same class (same subjects) In three dlffer- 
e. ; experiment?! treatments Is not, of course, possible In real life. How- 
ever, the computer Is capable of sett1n9 each, person back ♦<> his pre-test 
status at 'he end of each treatnent run! This feature provides a convenient 
method for the Garre Manager to discover the "real" .icC i e of the treatment 
effects. We would not recownend Its use In training researchers/eval uators 
— although it has some merit as a means of demonstrating the insidious ef- 
fects of testing error. 

Options. One unique option used in this request ijthe facilHy for at- 
trition (absence of subj|cts due to moves, quittlhg school etc.) over 
the course of an experiment. 

Commen ts. Note that even though the same class is us.ed, different sub- 
ject; are absent during different treatments. This happens because the 
absences are determined randomly within each treatment. (See treatment 
mitputs displayed on the follow; g tHree pages.) 
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4» Familiarizing Personnel with the Pract'cinn Process 

Having f air.i 1 i ari zed oneself with the operation of the FEHR data 
generator* the next step is to become familiar with the process of "solving" 
the REMAR problem. The most effective wa^y of doing this is for the prospecti 
game manager, consultants and associated instructors to complete each *ask in 
the practicum as a team, using only the Player's Instructions for guidance. 
Since the goal is to become familiar with all aspects of the system, we recom- 
mena that every task be completed— even those which you do not (presently) 
plan to use with the player- trainees. The only shortcuts which we would 
recommend are tnat only an abbreviated search of the literature be conducted 
and that al< written material be left in rough outline form. 

The amount of time required for a complete practice session varies 
widely with the user's training and experience in research/evaluation, and 
with the level of service supplied by their VocaTrcofflputer^^:i^articularly the 
turnaround time on computer runs. But on the average, assuming that the 
users are familiar with the local computer facility and the FEHR program, 
and that at least one team member has had sufficient training and expeHence 
to teach research/evaluation methods, the problem-solving activities per se 
should not take more than five or six hours. However, if computer turnaround 
time is slow, it will probably be necessary to schedule for three or four 
li to 2 hour meetings rather than one long meeting: 

5. Choosing a Place to Meet 

The most important activity during a practicum is the team meetings. 
The most satisfactory arrangement is a room or rooms with enough tables and 
(movable) chairs so that each team can have its owr table. Preferably, these 
should be far enough apart to permit comfortable, normal -voice discussions. 
It is best if there is shelf-space for storage of the Information Banks, 
1ST units, and other 16cal references. Similarly, a rcom which is near > 
laboratory facilities such as calculators, keypunches, computer terminals, 
and the like is much to be desired. Convenient access to the computer 
facilities is, of course, necessary for most usages. (See the examples of 
. FEHR usages for illustrations of suitable facilities.) 

There are several occasions during the practi -urn When it is dbsirable 
to have all participants at a single meeting. Because of the group activitii 
involved, it is best if the workroom used for team meetings is large enough 
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for this purpose as well. However, if such a room is not available, it is 
quite satisfactory to schedule the large meetings in a regular classroom 
nearby. 

p 

6. Selecting Materials for the Session 

There are, of course, several printed materials which are mandatory for 
aiv FEHR-PRACTICUM session. The title of each required material, its type 
(whether it is reusable from session to session or consumable), and the number 
of copies needed for each session are summarized below: 



TlrUr 



typt 



FEHR Materials Needed 



NO. COPIES 



Player's Introduction to FEHR-PRACTICUM 
Fair City USA 

Pi=»yer's Instructions for FEHR-PRACTICUM 
RFP Document (for the problem being used)* 

-*Wote:_lf the jjistruaions and RFP's 
have been used previously ,~chect to 
see whether the following inserts 
are needed: 

Replacement Forms for Instructions, 
pp. 2,4,6 



Replacement Checklist of Tasks for RFP. 
pp. 3-6 

Information Bank (for the problem being 
used) 



Reusable 
Reusable 

Partially 
Consumable 

Partially 
Consumable 



1ST Unit I. Assessing Success for Com- 
plex Objectives. . , - 

;t un:^ II. f^oi^'ffife 

1ST Unit III. Computer Format Statements 
for FEHR Data 

1ST Unit IV. Sampling the Subjects to be 
Studied 

1ST Unit V. using the FEHR Secretary 

*Note: The number of copies needed 
vanes with the session purpose. 
From 1 or 2 copies per installation 
to 1 copy per team. (See examples of 
FEHR applications for illustrations.) 

Game Manager's Manual for FEHR-PRACTICUM Reusable 



Consumable 

Consumable 
Reusable 

Reusable 
Reusable 
Reusable 
Reusable 
Reusable 



1 per player 

1 per player 

1 per pla^yer 

1 per player 



Included Above 
Incluoed Above 



1 per player 



In addition to the materials listed above, there are many cither 
materials which may be useful. These must be selected to match the needs 
and resources of the local area. However, the three following types should 
be considered. 

(1) If the content area of the problem is of substantive interest 
to the pi i^yer- trainees, copies of the primary references used 
in developing the problem may be desirable. Lists of the 
primary references for each problem appear on pages to 
of this manual. 

(2) If the published tests used in the problenL-are of substantive 
interest, it may be desirable to obtain copies of the tests 
and test manuals. 

(3) A certain number of research methods references should be 
available in almost all practicum settings. Fairly extensive 
listings of possible references are given in Appendices I and II 
of the Player's Instructions. 

7. Choosing the Practicum Tasks 

The first decision in the task selection process concerns the overall 
structure of the problem to be assigned. That is, should each team complete 
an entire practicum, only the descriptive phase (tasks 1 to 4 inclusive), or 
only ^he comparative phase (tasks 5 to 7 inclusive)? Considering the entry 
skills of the participants, the resources^avaiTabTe tin both material and 
personnel), the amount of time available for the session activities, and 
the session's overall instructional goals, which of these three structures 
should be used? 

Next comes a series of decisions related to the specific tasks to be 
done within the general session structure. The game manager's notes in 
Section II, Chapter 3 (Player's Instructions) of this manual are intended 
to guide users in making choices among the various options on the checklist 
of tasks. However, the set of tasks chosen for a particular session ought 
to be determined by the specific purposes the session is intended to serve. 
This relationship is illustrated in the workshop, module, and laboratory 
examples which follow these guidelines. 

8. Planning the Instructional Activi ties 

Throughout this manual we have maintained the position l^at since 
FEHR-PRACTICUM is not a self-contained instructional system, it is essential 
that the practicum be carefully articulated with an appropriate training 
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program. Depending on the circumstances, an appropriate training might 
consist of a set of independent study materials, a research/evaluation 
course built around the practicum activities, a set of highly skilled con- 
sultants available pn a tutorial basis, or various combinations of these 
three elements. 

* 

It is obvious from the above discussion that the number of possible 
instructional arrangements are limitless. We can not possibly provide 
guidelines which would anticipate all possible F£HR- PRACTICUM uses. In- 
stead, we have chosen to describe a few disparate applications in considerable 
detail. The workshop, module, and laboratory examples provide practical 
illustrations of how these tr^ifting arrangements can be adapted to meet specific 
needs. The details of these examples appear following these guidelines. 

Some Philosophical Considerations . It was stated previously that FEHR 
was designed to be a flexible pedagogical tool adaptable to many instructional 
purposes. To accomplish this aim,' the i^roblenis were described in ratner global 
terms, leaving the operational specification of the problem to the users. 
Thiis, If an instructor/Game Manager desired his/her trainees to practice pro- 
blem definition skills he/she could require the teams themselves to operation- 
alize the problem. If, on the other hand, the instructor/Game Manager wished 
to concentrate on research design and analysis skills, he/she might provide 
an operational definition of the problem and ask the teams to work within it. 
Additional adjustments to the ^cope of the players' tasks could be made by' 
restricting the number of treatments to be assessed, the number of variables 
to be considered, and/ or the number and type of research subjects to be used. 

Despite the conscious emphasis on adapting to an instructor's pur- 
poses, it would be a mistake to assume that FEHR is completely non-didactic. 
Like most instructional products, the FEHR-PRACTI CUM system is an implicit ' 
operational statement of the instructional philosophy of its authors. There 
is a pervasive bias which tends to nuture a particular view of the research 
process and to encourage the use of some instructional practices while dis- 
couraging others. We believe that the optimal results can be achieved only 
if FEHR is used in a manner consistent with its basic structure. Consequently, 
the remainder of this section is devoted to an explication of the more 'impor- 
tant beliefs and principles upon which FEHR-PRACTI CUM is based. 

Hz 
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i. We btlleve that the empirical evaluation of educational p.*o- 
grams is inherently a multidimensional process requiring the 
interrelation and synthesis of f*^quently conflicting information 
from a variety of sources. In our view, a single measure can 
almost never provide adequate assessment of educational effective- 
ness fieiSi- I" addition, the practical realities dictate that 
mai\y factors other than a program's effectiveness in meetviig an 
objective be considered. For example, the cost of a program and 
its degree of support among teachers, parents, and stlidehts must 
be taken into account. To complicate the process still further, 
there is always a host of irrelevant variables to divert the 
researcher/evaluator's attentlpn from the important issues. In 
an attempt to capture some of this multi dimensionality, each 
FEHR problem contains a variety of variables (tests) in each of 
several domains (attitude, achievement, etc.), and several sub- 
groups of subjects. 

b. We are firmly convinced of the validity of the Bush and McConnell 
(1966) findings that we best learn research skills by doing re- 
search. In the area of. evaluation and decision-oriented research, 
we would put the case even more strongly. One can learn to handle 
ambiguity and complexity only by working with ambiguous problems 

in a complex environment. Each FEHR problem is designed to provide 
this kind of experience. The problem definition supplied in the 
RFP is purposely broad and somewhat ambiguous, and there are 
always several treatments, many dependent variables (variables 
which change as a result of a treatment), and many moderator 
variables (variables which do not themselves change as a result 
of a treatment, but which change the effect of the treatment on 
one or more dependent variables). 

c. We recognize that for novice trainees it majrbe pedagogically 
desirable to begin on a sin^lified problem. However, we con- 
sciously opted not to provide simple problem descriptions providing 
only two or three treatments and a single dependent variable. 
However, the Game Manager or the players themselves may delimit 
the problem to provide an equivalent simplifying effect. It is 
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our belief that a problem which Is consciously delimited In 
the presence of complexity provides a more valid view of re- 
search and develops skills which are more likely to generalize 
to field research than would result from presenting only the 
delimited problem without the surrounding details. 

d. The above view of the reseaVch/evaluatlon process suggests that 
there is no universal research method which can be learned in a 
relatively simple context (e.g., a laboratory), and later applied 
directly to practical problems in a variety of settings. Rather 
there are a variety of methods and techniques which must be com- 
bined, adapted, and synthesized to meet the idiosyncracies of a 
given practical problem. Since these combinations and adaptations 
frequently result in methods which differ in substantive ways 

from the originals, we call the resultant strategy an idiosyncratic 
research method. 

The FEHR system provides for training in the development of 
Idiosyncratic research strategies in two ways. First, the eight 
i>roblems require vastly different research approaches. Second, 
within each problem it is possible to define the research objec- 
tives in several different ways, with each definition requiring 
a different research approach. 

e. The need for idiosyncratic methods demands that programs to train 
researchers/eval uators emphas"-?e the process by which a research 
strategy is developed rather than the strategy £er se. For this . 
reason the entire FEHR system is designed to create the desire 

to know and to provide an opportunity to discover. 

One can best illustrate the discovery approach by examining 
its alternative. It is possible to use FEHR didactically. For 
example, a particular research strategy could be taught by "solving" 
a FEHR problem in class, and then asking trainees to practice that 
solution method using a different sample of subjects (e.g., a 
different school). While this sort of practice is undoubtedly 
useful, we do not believe that it takes full advantage of th|^^ 



system's power. Nor does it facilitate learning how to ac^pt a 
theoretical method to a practical need. 




151 



144 



A less didactic procedure which Is mor^ cons Is tent with 
the training needs would proceed as follows: Flt^arC^ralnees 
are allowed to struggle with a problem until they develop a 
need for tlte method to be taught (but not long enough to become , 
overly frustr«lted). Second, the research method Is taught utilizing 
an example d1 fferent from the problem with which trainees are work- 
ing. Third, trainees adapt the method to their own problem needs. 
We are convinced that this ''discovery" approach will result In a 
greater depth of understanding and longer retention than more 
didactic procedures. 

f. The discovery approach outlined above req[u1res that a great deal 
of Individualized Instruction be available during the practlcun. 
The FEHR consultants are Intended to provide this service. In 
our experience. Intensive team-by- team consultation provides far 
greater Increments In learning than a comparable effort expended 
In large-class session— even though the latter method covers (at 
least superficially) far more material. To supplement the con- 
sultants, some users may wish to make a variety of programmed 
materials on research methods available to the players. Several 
examples of suitable materials are listed In the appendices of 

the Player's Instructions. In any case, we believe It Is a serious 
mistake to use the FEHR system to supplement an existing research/ 
evaluation course without adopting appropriate Instructional 
techniques. 

g. The foregoing emphasis on multldlmenslonallty and complexity en- 
courages teams to devise studies Involving data sets which are 
considerably larger than those found In the usual laboratory 
exercise. The opportunity to develop skills In this area Is a 
feature of FEHR which ought to be exploited whenever possible. 

h. Despite FEHR's admitted bias towards large data sets, the sheer 
size of the research populations, the number of available vari- 
ables, and the redundancy of information (e.g., some problems 
have seven or eight Intelligence tests) encourages the use of 
sampling for both subjects and variables. In most settings we 
would urge the game manager to provide further motivation in this 
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direction by placing reasonable limits on budgets* number of 
subjects, and number of variables. 

1. The budgeting aspects of FEHR are considered an Important and 
Integral part of the practlcum. More than any other element, 
costs motivate the players to plan their activities. Budgeting 
financial resources generalizes to budgeting of time and (non 
financial) resources. In fact, it has been our experience that 
the various costs attached to treatments causie troinees to change 
their behavior even when they have been told to ignore costs. 
For example, most trainees refused to use the Stanford-Binet IQ 
test when they noticed its price ($12.65) even though they were 
not being charged for it. ' 

In respect to costs, it is important for the game manager 
to realize that there is an intricate non-linear refationship 
among test cost, reliability and total experimental cost. The 
experimental costs can only be compared by holding statistical 
power constant. To get an Intuitive feel for this relationship, 
assume a matched experimental design in which there is a perfect 
correlation between the true scores of the matched elements. In 
this case, all the experimental error is attributable to test un- 
reliability. Thus, given test A with a reliability of .91 for 
$3 and test B with a reliability of .84 for $2. Using test B the 
error variance would be /l-r^ or 4/3 times the error using test A. 
To maintain statistical poWer equivalent to that obtained with 
test A, we must use 4^/3^ or 1.77 times as many people in the 
experiment. Thus' using test B we would actually spend $2 X 1.78 » 
$3.45 for each $3 using test A. 

j. Finally, we believe that the team approach provides an added 
dimension of great value to the FEHR-PRACTICUM experience. The 
value is of two sorts. First, our experience shows that there is 
a tremendous amount of intra team teaching and learning during a A 
FEHR project. Second, evaluative research in the practical worldf, 
tends to be a team project. Consequently, any group-process skills 
learned during the practlcum will have a direct and positive carry- 
over. He urge instructors/game managers to use teams, wherever 
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possible. Our experience shows that the team size should not 
be smaller than three nor greater than five. Urger teams tend 
to break into subunits with one set of trainees doing mfist of 
the work. Smaller teams tend to have less verbal interaction and 
hence less opportunity to learn. 

k. ThroughoutJthe^iicticum, every attemprt is made to keep the player 
so busy finding an educational answer that he or she has no time 
to woi^ about the( necess T^>9f learning the analytic skills 
,<^ietf^i«^y)to obtain that answer. This is the reason, for exwnple, 
^fer requiring that each player complete orientation question- 
naire at the very first meeting. Ih our experience, this has 
proven to be the most powerful feature of the model— particularly 
when FEHR is used with research novices who are fearful of taking 
a statistics course. In a field evaluation study/simllir ta the 
mod ule exam p le d esc r ibed laUji y-th^eet4en, literally dozens 
of students told the author that they got so involved with the 
problem they forgot to be fri^tened of statistics. 
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Instructional Options . A num^r of FEHR's optional features have direct 
instructional value. These options are of two kinds: those provided by the 
con4)uter program, and those external to the program. At this stage in planning, 
the instructor/game manager ought to specify which features will be used 
during the session. A listing of the features provided and a grief illustra- 
tion of the instructional purposes each might serve is provided below; 
«• Program options . In addition to the design and sampling options 
available to the teams themselves, a number of options are provided 
Oil card 1 of each request which are of great instructional value. 
Since the functions of the various parameters on card 1 are not 
described in the Player's instructions, these options are under 
the direct control of the game manager. We strongly recommend 
that each team be provided with punched cards for all system 
commands and card 1. Whenever the treatment multiplier option is 
used, the multiplier car. should be provided as well. The effect . 
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of each option on the program output is described- In the game 

manager's notes page . The Instructlonirt function of each is 

illustrated below. \ . 

i. Suimary statistics are esj^cially useful for classroom 
demons tratiOTS in which the patterns of survey or 
treatment differences «re to be illustrated. However, . 
. these purposel^ do not provide syfffci^t accuracy for 
precise significance tests. TKe standard deviations pro- 
vided are, biased estimates calculated using N degrees 
of freedom rather than K-K 
i i . Punchcard output vs of course veiry useful whenever the 
• output is to be analyzed by a canned computer program. 
It also has the addUional advantage in ^hat card out- / 
puts can be used to Identify subjects in (subsequent) 
requests to the data aener^u tor. For example, suppose 
a survey was run and t^.j output punched on cards. These 
cards could then be .usetd for the random selection of 
treatment groups, saving the bother of repunching the 
lU's on cards. HoweverV if this feature is used the 
subject parameter (card i colunh 5) must be 1. 
iii. Budget abor t. The budget\abort parameter allows the 
game manager to de-emphasize the importance of costs, 
iv. Attrition . The attrition parameter permits the game 

manager to control the complexity of statistical analysis. 
For example, zero attrition mqht be used when analysis 
of variance was first introduVed, but turned on when 
unequal M designs were studiea. 
V. Treatment multiplier . The option' to change both the 
absolute and relative sizes of treatment effects Is 
very useful when one wishes to demonstrate concepts of 
statistical power. \ i 

Non-program options . Several features of great instructional sig- 
nificance are offered outside the computer program. The most impor< 
tant of these are the message generator, the\ negoti ati oh of plan- 
ning grants or porposal "contracts",, direct team - computer inter- 
actions, and budget administration policies. \ 
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^* Message Generator. /TKe Xem message generator Is perhaps 
• a misnomer for the feature as It presently exists In FEH8. 
Originally we had planned to store a large n\BB|j||r of messages 
which could be r'andomly selected during play (or^-sel'ected) 
so as to simulate the Influence of external "Act/ of God" 
such as a hurricane which closes the scho6fs forSwo weeks, , 
or a teacher's strike with the same result. An example qf 
such a pre-selected series of messages appearsMn Appendix 
at the end of this manual. Game Managers are Invited to 
develop adaptations to meet tlieir a^n local needs. 

However, c<ur experience during the field trials was that 
the most useful messages were those that were sent with a 
specific didactic purpose In mind. , For example. In one 
REMAR session th^ players were universally opposed to using 
"error rpone teacher-made tests", consequently making all 
/ ^ selections and decisions on the basis of ^tajdardized norm- 
referenced tests and ignoring the criteri ©Referenced tests. 
, The following message was sent to each participating team. 

To? . REMAR ^Vearch Teams | 
From: A. Rumnus, Supeiintendent . 
Subject: Supplemental report 

I have recently received a supplemental report from the conmittee of 
math teachers. This statement indicates that pupil -performance on the 
teacher-made tests they have developed is of major importance. Standardized 
tests are usefuh but the teacHer test is a mastery test and gets most spec- 
ifically at what students should know. We believe that every grade 7 student 
should be capable of obtaining a perfect score (ID) on all four of these 
tests. H^lpifjg students achieve th1S'goa,l is the primary objective of any 
remedial program. 



In another case, the staff became concerned that teams were 
s-pending too much time in discussions without coming to grips 
with the specific tasks to be done. The following message 
was sent. 
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to: * . REMAR Research Team 

From: "A.' Lumnus, Superintendent of Schools 

Subject: Status Report from Research Team 

The Fair City Board of Education will meet this week-attf. according 
to their Chai man. Mr. Ontop. one Of the items on their agenda will be the 
research proposed to remedy-computational, deficiencies In the city's ^ 
seventh graders. ' He- expressed some concern about the amount of time spent 
on experimentation. Some members of the community are Impatient for action. 
He's quite sure that these persons will be in attendance at the board meeting 
and would like to be prepared to answer their questions. 

For this reason I mu.t ask you to submit a report summarizing exactly 

, whafltas been accomplished to date. Please direct this report to me 
through the game manager. . ^ " 



A final' example concerns the use of the message generator 
Idea to control a potentially explosive Interpersonal problem. 
In one session a number of black students became concerned that 
another team was "out to prove the superiority of Fair City 
whites". The game manager seized this opportunity to send . 
the "life like" memo displayed below. It Is suggested that 
similar memos can be developed by each game manager on- - ^ 
the-spot to handle incipent problems or provide didactic 
direction. 
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To: REMAR research/ teams * . ' 

From: A. Lumnum Svperintendent 
.Subject: Protecting civil rights 

A number of parents have recently complained thav many of our programs 
are sexually biased (e.g.. home economics). " There also appears to be a 
good deal of public sympathy for the notion that some of our programs are 
racially biased. Consequently, the Board passed a resolu;;cn stating that 
no/udent can be excluded from a program nor compelled to attend a (com- 
pj/lsory) program on the basis of sex or race. 

, In the interests of good public relations I have directed that In all 
/research projects presently under way the sex and race variables are to be 
ignored. No comparisons ,by sex or ra(^ are to be made, and there shall be 
no Identification of sex or race In.any research reports. Questions involving 

157 -^50 



identification by sex or race are to be omitted from all future file 
searches, surveys, or experimental treatments. 



11. Negotiation of funds . Rather than having a fixed maximum 
budget vf or pi arming grants or research proposals, It is 
possible to acccTWodate differences ,in needs and interests 
by negotiating the budgets with each team individually. 
This procedure has *)een very popular with both students 
and instructors/ game managers. It provides an ideal 
opportunity for checking design validity, sampling efficiency, 
logical sequencing of tasks. Some arrangements which have 
been successful include team negotiation with the game 
manager (instructor) in a single sess.:n, peer revi^i . of 
proposals by one or more (other) research teams prior 
to a negotiation meeting with the game manager, and tne 
u^e of the consultants in a session as a formal review 
panel. In all cases teams have reacted positively to the 
opportunity to "make their case", 
ill. Budget administration . The standard procedure for budget 

administration requires the game manager to act as a "Sank", 
and keep track of each teams account. Teams expend moneys 
by making out a check. Copies of the game manager's record 
form and a team's FEHR checkbook appear in Appendix at 
the end of this manual. 

If there is a great deal of work for the gome manager 
in a particular session, an alternative to ignoring the 
budget entirely is to ask teams to provide their own itemized 
record. An illustration of this approach is provided in the 
laboratory example ^example 3) presented later in this 
section. 

1v. Compjter-team interactions . There are two ways in which teams 
are required to interact with the computer. The first is the 
submission of a request to the FEHR data generator. In the 
usual case, a team will need to prepare a r^:.imum of three or 
four separate runs during an entire sesi.on. These could 
easily be handled without requiring the players to do dii^thinj 
more than punch a few reauest cards. 
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The second type of interaction y^ith the computer comes 
about when a team analyzes the FEHR output with a canned 
computer program. The rfumber of encounters Involved in this 
process is abviously highly dependent on the local 'Computer 
system, since no analytic programs are Included in the FEHR 
package. However, it has been our experience that the com- 
puter exposure involved in using the data generator mini- 
mal as it is — has a positive impact on student's perceptions 
of the computer, and their willingness to learn Movi to use it. 
We strongly rr:ommend that instruction in the use of canned 
programs at your local computer center be coordinated with 
^ the practicurtu In this respect, 1ST unit number III, "Com- 

puter Format Statements fc^ FEHR data" should prove usefuK 

9. Evaluating the Practicum Products 

There are five team products described in the checklist of tasks to 
be accomplished: (l) daily log, (2) request for a planning grant, (3) formal 
problem description, (4) proposal, (5) final report. (Another product, the 
orientation questipnnaire, might be used as a pre-test or benchmark, but should 
not be evaluated as a session cask.) 

These products—especially the last three—provide definitive evidence 
of whd^ the team has done, and, by inference, what the teams can do. However, 
the extreme flexibility in the form and content of the products preclude a \ 
meaningful discussion of evaluation methods at this point. Rather, we have 
chosen to provide specific examples of evaluation in sessions with differing 
purposes. These are included in tlie descriptions of thfe workshop, module and 
laboratory examples which follow these general guidelines. The item is in- ; 
eluded here to emphasize the need for some product evaluation to be included 
In every session plan. 

10. Evaluating the Effectiveness of the Practicum 

The final step in organizing a practicum session is to plan some means 
for assessing the instructional effectiveness of the '"vtal session.. These 
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will hoptfully Include both the perceptions of those Involved arid some ob- 
• jective evidence of learning. Aga1n» the flexibility in usage precludes a 
meaningful description^^ evaluation methods at this point. This item was 
included in these guide Ities to emphasize the necessity for doing evaluation. 
However, some specific suggestions for evaluation are included in the examples 

which follow, l! 

i 

EXAMPLE 1. THE IN-SERVICE WORKSHOP 

Mr. X was an adml tiator for a regional laboratory for educational re- 
search and developmen He had just received funding for a large nev^ project 
which required the development of a series of proposals for evaluating program 
components. He was presently interviewing applicants for approximately twenty 
new staff positions. Although most of the applicants had a good grasp of the 
b»sic principles of research design and statistical analysis, they were all 
quite weak in what he called the applied skill areas: problem conceptualization 
quasi -experimental design, budgeting, determining cost effectiveness criteria, 
and so on. 

1. Defining Ses&ion Goals 

After reading Sections I and II of this manual, Mr. X decided to hold an 
intensive two-week FEHR-PRACTICUM session which would be compulsory for all 
new personnel, and voluntary for present staff (excluding project directors). 
However, to encourage attendance of the jresent staff he had let it be known 
that three new project directors would be chosen largely on the basis of work- 
shop perfonnance. Ten of the present staff voluntarily enrolled, providing a 
total of 30 player- trainees for the session. 

Mr. X specified his end-of-session goals as folloWs: 

(a) Every participant should be able to: \ 

(i) State a problem in terms of ; elationships among^ variables. 

(ii) Prepare a budget which accurately summarizes the costs of a 
proposed project in an acccfptable form. 

(iii) Develop a (quasi-experimental?) design which controls for errors 
introduced by factors such as tie required use of intact groups 
or differential attrition among treatment groups. 

(iv) Develop and defend a criterion of success for determining the 
degree to which a given objective has been met. 

(v) Function smoothly in a team resetrch effort. 
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(b) In addition to the above, the session should permit selected par- 
ticipants to be judged with respect to their ability to: 
(i) Select personnel . 

(ii) Plan and organize an efficient team effort to meet a given goal. 

(iii) Synthesize the Ideas of several group members. 

(iv) Respond calmly and effectively to unexpected administration 
requests to justify a decision or to modify an ongoing project. 

2. Choosing Personnel for the Session 

Hr. X decided that he would delegate the actual supervision of the session 
to Mr. Smith, his administrative assistant. Smith was not a particularly strong 
researcher, but he was a good organizer, a meticulous bookkeeper, and had had 
extensive experience with the local computer. Accordingly, Smith was appointed 
Game Manager, with the understanding that Mr. X was to be consulted in all in- 
structional decisions. 

The choice of consultants for the session was particularly important, 
since Nr. X could find no ready-made courses as independent study materials 
well -suited to his training purposes. However, there were three highly skilled 
project directors available. These people were appointed as research consul- 
tants for the session. To ensure maximum usage, it was decided that consultation 
would be free and scheduled rather than the paid service described in the 
FEHR-PRACTICUM descriptions. 

3. Familiarizing Personnel with the Data Generator 

Since they were using FEHR for the first time, Mr. X, Mr. Smith, and the 
three consultants ran through each of the familiarization requests just as 
they appear in the general guidelines. 

4. Familiarizing Personnel with the Practicum Process 

Mr. X and his staff worked as a team to go through the entire practicum. 
They completed every task except #7, the consoli ' >n session, v/hlch they 
d4d not complete since they planned using only the tasks up to and including 
the proposal (but excluding running the experiment or completing a final report). 

5. Choosing the Place to Meet . 

Mr. X decided that their regular staff meeting room could be used for 
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-'tht larga-group sessions. Tttrteons could meet In individual offices much 

.as they 4\6 in thetr dtty-to-d^y work. Since two key-punches and a small 
^ library were available' In the office comrlex, as was a card reader connected 

to a remote computer shared with the local school district, this arrangement 

was considered entirely satisfactory. 

Selecting Materials for the Sesslo: 

7he regular fEHR materials listed on page (3-21) of this manual were 
ordered, with three copies of all materials marked with a* double asterisk. 
Copies of most of the reference materials listed in appendix I of the Players 
Instructions were already available In the existing library, along with a 
few standard statistics refer^n:es. These were considered adequate for the 
purposes of the proposed practicum. 

7. Choosing the Practicum Tasks 

It was observe'', that the session goals put heavy emphasis on problem 
definition, operational 1 zing relationships, budgeting, and designing^ but 
little emphasis on statistical analysis and interpretation. The latter 
were areas in which most personnel were fairly skilled already. Because 
of the time constraints, Mr. X decided to assign only tasks 1 to 6.1. 
The specific assignments appear on the checklist pages to 

8. Planning the Instructional Activities for the Session 

At the beginning of the session the Orientation Questionnaire will 
be expanded to Include (on the back of the Questionnaire) a brief narrative 
description of the player's view of the problem and his/her proposed research 
strategy. These will be ranked in order of quality by the three research 
consultants. The players having the top ten questionnaires will then become 
team leaders. The leaders will be responsible for recruiting two other 
players to serve as teammates. 

The major instructions", activity during the session will be scheduled 
(free) con§i.''tat1on. Throughout the practicum each team will receive two 
full hours (two one-hour periods) of direct tutorial consultation each day. 
Consultants are rotated to give each team equal exposure with each. Mr. X 
will meet with the game manager and the three consultants and for a one-hour 
brown-bag lunci.ion each day. At these sessions the progress of each team 
and of each individual is to be reviewed in detail, and various instructional 
strategies suggested and discussed. Mr. X will keep annotated records on 
the progress of each individual. Occasionally he will ».sk consultants to 
makk specific observations for a few individuals for whom the evidence is 
Inconclusive. 
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9. Evaluating the Practicum Product s 

Mr. X and his associate director (Smith) have both evaluated large 
nunbers of proposals for the U^S. Office of Education. They plan to evaluate 
both the problem definition (task A A) and the proposal (task6.1) using 
USOE standards. After each evaluation one or the other of them plans to 
hold a half-hour session with each team discussing the strengths and weak- 
ness of the product evaluated. 

10. Evaluating the Instructional Effectiveness of the Session 

The main evaluation of individual learning will occur through evaluation 
of the team products and the observat .lal records kept by Mr. X. The new 
project directors will be chosen on the basis of this information. The 
aggregate amount of observed change in performance by all players will be 
one method of evaluating the session as a whole. In addition, each pls^yer ^ j 

win be asked to complete a detailec jestionnalre rating the value of the 
session experience. But the most crucial information is the subsequent on- 
the-job performance of the participants. Mr. X plans to keep detailed 
records which will allow him to compare the performance of FEHR participants / 
with that of non-participants who were previously on staff. If the evidence 
Is supportive, he plans to have a subsequent FEHR - practicum concentrating 
on the comparative phase and emphasizing statistical analyses. 

EXAMPLE 2. THE RESEARCH TRAINING MODULE 

Mr. Y was a professor of educational research who taught a three-course 
series in research design and data analysis which was a pre- dissertation re- 
quirement for most Ph.U. students in Education at his institution. He had \^ 
noticed that many students who did quite well in his courses were completely 
unable to fomulate an acceptable dissertation proposal or conduct the subse- 
quent analyses and interpretations without extensive individual consultations 
of a tutorial nature. What was needed, he felt, was more direct Instruction 
in the applicatioti s^f the principles developec in class to a practical problem. 
At this point Mr. Y went on to steps 2,3, and 4. The detailed goals for 
the session were actually written after he had become thoroughly familiar 
with the practicum groups. For reading convenience both broad and detailed 
goals are listed here. 

1. Defin1n3 Session Goals 

Mr. Y believes he might best train for application by using a FEHR 
problem as a core experience and reorganizing the content of h!s previous 
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tmPli I. THE IN-SERVICE WORKSHOP 
CHECKLIST OP TASKS TO BE PERFORMED 



3 



Assigned 
Tasks 



Description of Task 



Instructions 
Page 



(1) 



(2) 



(3) 



Each player should submit an Orientat ion i 
Questionnaire to the Game ManigH: — ThT" 
specific question to be answered are in 
the INSTRUCTIONS. 

J^f^riTf^-^^ directed by the Game Manager. 1 
Ttie following assignments and requirements 
apply throughout this PRACTICUM: 

Team salary: $JCQ per day. 

Costs are to be paid as indicated below: 

(a^ ignore all costs 
.^{b) pay costs by check 
(c) pay costs with computer receipts 

Requests for file learches. survey and treat- 
ments should be? 

«"teJ^ed by the team via PROMPTS 
^(b) entered by the team via punch cards 

(c) written on the appropriate fonn and 

submitted to the Game Manager 
A Log of research activities: 

(a) to be kept by each player 
./^(b) to be kept by each team 
___(c) no long necessary 

iSS", ««"?ctiSn.?"''''" Observing the fol- 3-5 



Literature: 



Definition: 



n o review necessary 

Information Bank only 

- )^.Inforniation Bank and regular 
^library 

> Toblem definition provided 
.by Game Manager 
^^complete problem definition 
fozm 
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ifasks Description of Task ^^j^ 

y (4) Determining Extent and Severity of the 5-7 
Problem 



I Submit a Request for a Planning 
Grant on the form provided 

Due Date: E Oe ) Y^^^j ^ 

Maximum Grant $^ 



^ 4 .2 Run File Search and/or Survey ^ 

Maximum sample sizei C>A /ni.^ C^" C ^^S-^A 



Maximum number of variables_/^ 
( independent; ^ dependent) 



4.3 Analyze c the data from 4.2 using 

appropriate statistical procedures 
j^4.4 Formal Problem Description (first 
> draft) . To be presented ror cri- 
tiquing by the other teams on 

(date)__2^a^ ^ 

(5) Funds are available for identifying a Pool 7 
of Research Subjects for Treatments 

Yes- Xno poof ^i fl S^kc^ ' I 

(6) Empirical Evaiuction of the Ed(ucat£on^^ ^ C <tir<f^^ 
Treatments 

/ 6.1 Submit a Research Proposal to the 
Game Manager. The proposal mus'. 
meet these restrictions and re- 
quirements: 

Maximum number of variables per ob- 
servation 

Dependent ; Independent_^[-_^; 

Tota l /f^ 

Maximum number of observation 
times v'h 

Maximum number of research subjects 

Da^^roposal must be submit ted_^^j ^ - • 

The maximum project budget is: $ 1£. 

The proposal should cc tain each item 
checked below: 

X (a) abstract of the proposal 
"y" {b) introduction and overview of 
problcn 

o 
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Assigned 
Tasks 



Description of Task 



Ins tactions 
page 



X (c) a review of the related 
^ literature from: 

t he ^ Information Bank only 
Xb oth the Information Bank 
and library 
X (d) a formal statement of the problem 
_}5^(e) a description uf the procedures 
y to be used 
A (f ) a schedule of activities 
(q) an item: zed budget 

The writing style for the proposal is 
checked below: 

brief outline 
detailed outline 
formal narrative 




6 .2 Using an appropriate method, divide 
your sample into treatment groups, 
observing the restrictions given in 
Task 3. Run the experiment on the 
FEHR Data Generator. Data will be 
ojtput on: 

(a) printed copy only 

^ (b) printed copy and punch cards 

(c) printed copy and the team's 

computer file 

^6.3 Analyze an^v interpret the data frc»n 

Step 6.2 using appropriate statisti- 
cal procedures. '^ecord reatxictions > 
(to be announcet. by the Game Kahager) 
below: 



6.4 Submit a final report to the Game Mana- 
ger. The report should contain the 
proposal items checked in 6.1 plutf the 
items checked below: 

(h) the results of the field test 

(i) H summary of the statistical 

f indings 

(i) a discussion of results culmi- 
nating in an educational decision 

(k) a discussion of the limitations 
of the project 



0 
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Assigned 
Tasks 



Description of Task 



Instructions 
Page 



0 



(1) reconur.endations for future 

research 

(m) an appendix containing the 

Iog(8) of activities 

The writing style of the final re- 
port shpuxd be: 



. (a) brief outline 

(b) detailed outline 

(c) formal narrative 

Final report due date:_^ 



(7) 



Attend a consolidation Session (at a tune 
to be announced by the Game Manager) to Ci.s- 
cuss the strengths and weakn^sses of the 
strategies employed by your team, and any 
other teams which are attacking the same 
'iroblem 
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ftsurcK cUsses around the practtcum U^k^. , He decided to try-out fits tdea 

by using the RENAR problem as the core experience for the content previously 

covered by the first course In the sequence. The old course had covered 

the basic scientific principles underlying the research process and Introduced 

research dssign and analysis technlzues up to an Including a simple analysis 

of variance. Because he was convinced that his proposed shift In emphasis 

demanded careful analytic thought, Kr. Y spent a good deal of tir. : explicating 

his purposes. He defined his goals for the session as Itemized below: ' 

(a) The primary end-of-sesslon goal Is that each student can apply \ 
the research Ideas taught 1 17 class to a practical problem. In ) 
operational terms this was stated as follows. Given an example / 
of each FEHR-PRACTI CUM task, based on carefully-chosen demonstration 
variables, each student will, during the course of the session, ] 
demristrate his/her ability to apply the principles and procedures \ 
Involved by successfully completing the same task using the unique 
set of variables assigned to him/her. Each product produced In the 
above process should meet or exceed the standards of research 
quality comnonly used In judging dissertations. In particular, 

each student should produce a research proposal and final repor t which 

together demonstrate the followfng characteristics: ^ : 

(I) Non^amblguous, operational definitions. \ 

(11) A critical assessment of the meanlna of studies reported 1h 
the literature as opposed to an acrltlcal acceptance of their 
conclusions. 

(Ill) Efficient scheduling of activities. 

(1v) Careful budgeting of time and money. 

(v) Sampling procedures which ensure general Izablllty. 

(v1) Use of efficient and valid research designs. 

(v11) Descriptions of procedures which are precise enough to permit 
replication. ^ 

(vlll) A systematic assessment of error In making Inferences - par- 
ticularly when there are multiple comparisons or when i^ost- 
hoc analyses are usee*. 

(1x) Educational conclusions (decisions) logically derived from the 
evidence. 

. (x) The use of a style and format consistent with accepted scientific 
standards (e.g., APA Publication Manual). 

CbJ Although most of the knowledge and skills assessed In the former course 
are prerequisite to the performance of goal (a) above. It Is specifi- 
cally stated that each student will be able to: 

. (1) Identify and describe the steps in the research/evaluation 
process. 

(II) Distinguish between external and Internal validity. 

(ill) Describe and critically assess the following procedures for 
controlling coninon threats to validity: use of a control 
group, randomization, matching, randomized blocking, repeated' 
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neasures (subjects used as their own control). \ 

Civ) Constr. ct alternative hypotheses from a given priiblem des- 
cription. 

(v) Construct a null hypothesis for a given research (positive) 
hypothesis and describe its use. . \ 

(vi) Classify variables in given experiment as mariipulable (treat- 
ment), dependent moderator control, or intervening \variables, 
and describe the characteristics of each variable type. 

(vii) Construct operational definitions of given variable^. 

(viii) Construct criterion of success for a given educatioi^al objective. 

(ix) Construct predicted outcome patterns for experimental com- 
parisons which are consistent with a given hypothesis. 

(x) Describe and critically assess the following procedures for 
controlling threats to validity: • , ' 

(a) delineation of the research population • , 

(b) method of removal 

(c) method of constancy ^ 
td) use" of a control jgroup 

(e) randomization ! 

(f) matched pairs 

(g) matched groups 1 

(h) randomized blocks ; 

(i) using subjects as their own controls j 
(j) method of counterbalancing i 

(xi) Distinguish irnwg pre-experi mental , quasi -ex!;/erimental ^nd true 
experimentaV designs based on their adequacy for handling 
threats tflyValidity. i 

txii) e:orT^trQct true experiire.ital desigrts (up to simple 1-^toJt'tal) 
io'meet the needs of a given experimental situation. j 

(xiij) Describe a given study using the Campbell Stanley notadon. 

(xiv) Identify, describe, and critically accessthe utility of 
each of the following measurement device or concepts: 

(a) test reliability (3 types) ( 

(b) test validity. (4 types) , - / 

(c) scale of measurement (of a set of scores) 

^ ■ . (d) grade-equivalent (or age-equivalent) scores 

(e) ranks 

(f) percentiles 

(g) standard scores (several types) 

(h) comnon research scales (e.g., Likert, Semantic differential, 
* rating scales, Thurstone scales). 
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(xv) Conputp. for an appropriate set of supplied data,' each of 
the following quantities using a canned computer program or 
a calculator (at the student's option): 

(a) ' descrfptlve statistics Including reliability, validity. 
\ medium mean* and standard deviation for scores on any 
^ one test plus Pearsons rand Spearmans rhq.to describe 

relationships between different variables. 

(b) score transformations 'Including ranks, percentile ranks, 
and standard scores. 

'(xvl) Using canned computer programs, compute and Interpret the fol- 
lowing sIgnlflcaQce tests: 

(a) correlation ci^fflclent (greater than o7) 

(b) two correlation coefficients 
(cj t test for a difference In i^d^endent means 

(d) t test for, a difference In paljPM-score means 

(e) chl-square test for Independent ((roups 

(f) one-way analyst of variance 

2. Cho feing Personnel for the Session _ 

Mr. Y was fortunate In thaj^he had a teaching assistant assigned to help 
\ him with the laboratory section of his (old) source. TKe assistant, he 

decided, could act as game nanager (supervisor).' In addition.,, he decided 
to assign the course ass^arit the role of research consultant. However, 
research consultation was to be free and on a scheduled basis rather than 
a voluntary budget Item. Mr. Y himself would assume full responsibility 
for all Instructional functions. 

3. /amlllarlzlng Personnel with the Data Generator 

^ Mr. Y had his assistant prepare the cards for each of the five computer 
. runa^1vp«» In the general guidelines and submit the runs to the computer. 
At first there were a few problems In getting the program to respond, but 
with the help of a systems programmer at the computing center this was quickly 
corrected. Mr. Y and his assistant' then went over each inquest together, 
comparing their resLlts with those shown In the manual. They has one or two 
questions about the use of the card 1 options which were not satisfactorily 
covered In the manual, but these they were able to answer for themselves 
by running the same -request several times using a,d1fferent card 1, option 
each time. 
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4. Fawn 1 arizing Personnel witK tKe Practicuti Process 

Mr. Y'and his assistant conferred together i.o define the REMAR problem 
and outline the general research strategy. The assistant was then delegated 
the responsibility for completing the' first six tasks on the checklist using 
a brief outline style for all w 'tten materials. He was asked to keep a 
concise log which they could later use for scheduling assignments. When th.6 
six tasks were complete, Mr. Y and his assistant met to determine how they - 
night best use the decision subro;.-ine of the dat^ generator. (This sub- 
routine <s not available to player- trainees.) By .this time, Mr. Y had decfdfed 
that, for pedagogical reasons, 'he was going to restrict all teams to experimental 
-contrasts using standardi2e<i tests only - eVen though he believed that the 
attaininent^^of-~perfect_scores m all four of the CRT tests (addition, subtractiun, 
multiplication, and division) was the crucial criterion of success. There- 
fore he decided 1;o^use the Decision subroutine to compare each student's 
decision with the best possible decisions he/she could have made using only 
standardized -^**£ information. The entire process is described here to. 
provide guidance in using the Decision feature. - 

The first step in us^ng the Decision Subroutine is to define the population 
dto be used in assessing treatments. This "^target, population for the session, 
they decided, would consist of all REMAR subjects who had a score of zero 
on one or more of the CRT tests.. Since computer time was relatively expen- ^ 
.sive at their installation, a random sample of -200 of these students was used 
in the decision routine rather than the entire population. - 

The second step was to define the set of dependent variables to be used, 
and the relative importance of each variable. It was- decided. to use all seven 
standardized tests of mathematics achievement, with quadruple weight on the_ 
one test measuring computations directly and unit weight on the other six 
tests. The weights f6r variables 042, 043, 044; 059, 060, 062 and 063 are 
.4,.l,il,. .,.l,.l, and .1 respectively.- . . ' ,, 

. The third step was to determine for each variable the minimum and maximum 
cut-off points corresponding to "absolutely no success in meeting this objective" 
and "complete success in meeting this objective" respectively. Since the 
CRT tests were accepted as definitions of success/no successi the first step 
in this process was, to find operational (standardized test) definitions of 
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the point at which all CRT tests equal zero and ill CRT tests equal 10. 
Because the Items on eac.i CRT test are considered to be redundant, measures of 
exactly the samt; skill, Mr. i defined the success point on each standardi 2;:d 
test as^tne mean of the sccres of those subjects whose ' *i1 score on the 
four CRT test is above 30 and less than 40 (I.e., they ."ight ct the 
^/\x/ point of complete skill but not ^^t^ o^oj ^ JUtfe'te CKT ^ 
/ point on eaclK,t|^tj|as^ef 1 ned as^thfemean score of all those i(Rii.Lir>i i^- 
^ 1*55^6 A* ft CRT tests. Fhe^'^s jainy ii ininms ^^a«»"*o^ili«»»»P^I**t* 

mn) and niaxiniums ( i iMii #UH t l UiB imui l ^ ! . I II BTB "Jtm) are given U, the 
t%eU-below. 

Variable Index 042 ' 043 053 05u 052 D5I~ 



Mini mum 



Maximum 



The fourth and last step is to enter the Information tnto the computer 

according to the instructions given on page _^ . The card deck for this 

decision run appears on the opposite page. 

5. Choosing a Place to Meet 

V 

The classroom used for the old course was quite near the statistics , 
laboratory containing three computer terminals and a number of calculators. 
A small 11brary/wor.,*k3p was also available in the laboratory complex, "nese 
facilities were considered quite adequate. The scheduled meetings are described 
later, i 

6. Selecting Materials for, the Session 

The regular FEHR. materials were to be made available according to the 
list given on |». (3-27) of this handbook. Five copies of each 1ST unit and 
five copies of the REMAR Information Bank were ordered on the basis of one 
for each five students e^xpected in the laboratory sessions. (Fifty students 
were expected to enroll, but these were to be split into two laboratory 
classes of. about twenty five each.) 

The library in the laboratory containing a good set of statistics references 
.and a iocd sample of educational research texts. However, it was decided 
td obtain five sets of the independent-study materials develooed by Southwest 
Regional Laboratory as well. These materials are listed under references 
In Appendix I of the Player's Instructions (see page oi this manual). 
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7 Choosing the Practicum Tasks 

On t\A basts of thfe time his assli t^nt took to complete the six practlcurn 

task?, Mr- Y decided that his class coulc cover all the tasks only If the 

number of alternatives were sharpiy reduced. These re«;tr1ct1ons appear In 
the completed Checklist of Tasks on pages to 

8. Planning the Instructional Activities of the Session 

The old course had been scheduled for two class-sessions per week ' 
each of two hours duration. In eddltlon, each ^tudent was expected to . 
spend from two to four hours In the laboratory each week. It was decided 
that all the students In the new course would meet for a two-hour "lecture" 
each Monday}' ther) split Into two groups of about 25 persons each for two 
separate laboratory sessions, with group 1 meeting on Wednesday and group 
2 on Thitrsday. In the Mondsiy session Mr. Y will present the basic content 
of the course, with material presented In order of Its occurrence In the tasks 
to be done. Each task will be Illustrated using a set of demonstration 
variables. 

I r 

On Wedn-^sday and T^jrsday the students apply the same Ideas to their 
REMAR .problem. As Indicated prevlogsly, e^ch student will have a unique 
sample of subjects and a unique set of var'fables to study. Everyone will 
stuciy two dependent variables (the SAT com|i|utation sub-te and one other 
test chosen from the six standarlzed test:. \r\ mathematics), and one Independent 
(control) variable, under three treatment different treatment conditions. Tht^ 
elements to be studied will be combined so as^ to form unique experiments 
for each student. 

Throijghout the laboratory periods students will be encouraged to work 
In three-member teams to dlscu'^s process, but to complete each task on their 
own. Thus, there will be approximately six "teaVis" In each lab ses^on. 
Mr. Y plans to spend 15 minutes 1n consultation ^1th each team each lab period. 
The game manager (teaching assistant) will be on call during each lab period 
and available foi additional Individual help by apj^ointment for an additional 
four hours each week. \ 

Mr. Y feels that although he Is adding two additional contact hours 

each week, he will actually spend no more time on thi^ total course than usual. 

\ 

\ 
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\ EXAMPLE 2. THE RESEARCH TRAINING MODULE 3 




CHECKLIST OF TASKS TO BE PPRPORMED 



Since the work to be accomplished under this RFP varies from loca- 
tion to location, your Game Manager has^ placed a ( ) before the 
particv^lar items for which your team is responsible in the PRACTICUM. 
(If no items are checked, see your Game Manager before proceeding.) 
A detailed explanation of how to accomplish each task appears in the 
Instructions section on the page indicated at right. Tasks marked 
with an asterisk (*) are mandatory in every PRACTICUM. Before be- 
ginning, each player should study, interrelate, and synthesize the 
information contained in the RFP, its various appendices,, and the 
relevant parts of the Pair City Booklet. 



Assigned Instructions 
Tasks Description of Task Page 

» (1) Each player should submit an Orientation 1 
puestionnaire to the Game Manager. The 
specific question to be answered are in 
the INSTRUCTIONS. 

♦ (2) Fonn tesms as directed by the Game Manager. 1 

The following assignments and requirements 
apply throughout this PRACTICUM: 

Team salary: $ /OQ per day. 

costs are to be paid as indicated below: 

(a) ignore all costs 

X (b) pay costs by check 
(c) pay r^otwt with computer receipts 

R^uests for file searchers, survey and treat- 
ments should be: 

^a) entered by the team via PROMPTS 
Xl(b) entered by the team via punch cards 
written on the appropriate form and 
submitted to the Game Manager 

A Log of research activities: • 
X (a) to be kept oy each player 
y^ept by each team 
(c) no long necessary 

♦ (3) Operationalize the problen observing the fol- * 3-5 

lowing restrictions: 

Literature: n o review necessary 

^l" Information Bank e»4y p/ic^O/ie 
Information Bank and regular 
library 

Definition: Problem definition provided 

by Game Manager 

c omplete problem definition 

form 
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Assigned instructions 
TasXs Description of Task Page 



(4) Determining Extent and Severity of the 5-7 
Problem 

J^4.1 Submit a Request for a Planning 
Grant on the form provided 

IXie Date; Af>prriXifrisHeJ\J ^ej>h ^^ 
Maximum Grant' $ ^^ ^A^r rsj^/A -f^g/ 

X^^4>2 Run Pile Search and/or Survey 

Maximum sample s ize : \3 Z*/)'^-.^'/ c:JoSS&<k 
Maximum number of variables 

( \3 independent; £7 dependent) /f5S/y/>gy 

y 4 >3 Analyze the data from 4.2 using 

\. appropriate stjitistical procedures 

A^4>4 Formal Problem Description (first 
draft) • To be presented for cri- 
tique ag by the other teams on 
(date) ^ r> ^f^^'/».^-^>/fy Oc^ fS: Co/lrt^^^O^ 

(5) Funds are available for identifying a Pool 7 ^ 
of Research Subjects for Treatments 

Yes? X NO iA/zi'/ S\Jp^fltj6 01 

(6) Empirical Evaluation of the Education 7-9 
Treatments 

X 6 > 1 Submit a Research Proposal to the 
Game Manager. The proposal must 
meet these restrictions and re- 
quirements: 

Maximum number of variables per ob- 
servation: 

Dependent ; Independent / 
Total 



Maximur number of observation 
tiroe s f 

Maximum number of research subjects 

Date proposal must be submitte d //piy / gippto^ f^^i^/^j^ 
The maximum project 
The proposal should 
checked below: 



\t be submitte d /Jo%j / gt^pto 
set l^udget is: $ tJ^o ifJkl^ 
lid contain each ItSft 



(a) abstract of the proposal 
[(b) introduction and overview of 
problcn 
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Description of Task 



Instructions 
Page 



y (c) a review of the related 
literature from: 

t he information Bank only 
"X"both the infonnation Bank 

and HhmvT OnC* bf^tf 'SfV^%f 

i(d) a formal statement of the problem 
(c) a description of the procedures 
to be used 
^ (f) a schedule of activities 
"X (g) an itemized budget 
The writing style for the proposal is 
checked below: 

(a) brief outline 

-X (b) detailed outline 
(c) formal narrative 

2 Using an appropriate method, divide 
your sample into treatment groups, 
observing the restrictions given in 
Task 3. Run the experiment on the 
FEHR Data Generator. Data will be 
output on: 

(a) printed copy only 

(b) printed copy and punch cards 
"^ (c) printed copy and the team's 

computer file 

.3 Analyze nd interpret the data from 
Step 6.2 using appropriate statisti- 
cal procedures. Record restrictions 
(to be announced by the Game Manager) 
below: i ^, 

.4 Submit zi^ final report to the Game Mana- 
ger. The report should contain the 
proposal items checked in 6.1 plus the 
icems checked below: 

X (h) the result ii of the field test 
"^(i) ft summary of the statistical 
findings 

X (j) a discussion of results culmi- 

nating in an educational decision 

(Ic) a discussion of the limitations 

of the project 
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Assigned 

Tasks 



(7) 



Description of Task 



Instructions 
Page 



(1) reconur.endutions for future 

research 

J^^) an appendix containing the 
log(s) of activities 

The writing style of the final re- 
port should be: 

.(a) brief outline 
.(b) detailed outline 
.(c) formal narrative 

Pinal report due date: /(o.W/>..W^ / 



Attend a consolidation Session (at a time 

Manager? t^^dis- 
cuss the strengths and weaknesses of the 
strategies employed by your team, and any 
prSblem^^' attac .ng ihe sa^e^ 
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In the old course he spent several hoqrs eacK week checMng pn the nathemntcal 
•ccuracy of students' statistical problems. The new emphasis permits 
h1« to check ualy the accuracy of the entry Into the computer * he belu^es 
that he and his assistant can do most of this during the lab sessions. The 
additional emphasis on problem definitidn. measurement, design, and general 
research methods which are to be substituted for the calculator computations 
in the old course will be checked and reinforced during the consultation 
sessions, as well. The instructional power of of the whole system depends 
on regular feedback, 
g. Fvaluatinp the Practi cUtt Products 

The two main products (the proposal and the final report) will be 
subjected to both peer review and Instructor review. Peer review will 
be done on an assigned round robin basis usin^ a checklist supplied. by the, 
instructor. The Instructor wITl then rate both the product and the checklist 
for accuracy^ and completeness. Any incomplete or Inaccurate elements must, 
of course, be corrected. Before the end nf the course each student must have 
producU. which are acceptable in all respects. ^ 
10. Evaluating the Instructional If fectiveness of the Practicum 

The iwnediate instructional effectiveness of the practicum will be 
assessed in four ways. First, a brief achievement test will be given every 
three weeks. The applied statistics items from these tests will be taken 
directly from statistics tests of previous years. Performance on these items 
can be compared with previous years. 

Second, both the instrtictor and the teaching assistant will keep a checklist 
of skills. Each Jtudent will be observed regularly during the lab; whenever 
he is observed demonstrating one of the listed skills (e.g.. running a parti- 
cular planned program successfully), the skiU- will be checked off on the list. 

Third, the products of the sessions are considered to be direct evidence 
of students'ability to apply research ideas. In addition to the marking pro- 
cedure explained above, other faculty members will be Invited to examine these 
products and coirr-^-it upon their adequacy. 

Fourth, a questionnaire will be^ administered at the end of the session. 
It will Include a series of items asking students to assess their learning 
during the session. ^ 
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tXiwPU 3. APPLItiL STATISTICS LABORATORY 
Ms. Z was also a professor of educational research, but she taught 
only one senior course featuring the analysisof variance and covariance. 
Students had frequently complained that the course was both too ^heoVetical 
and too difficult and thSit the examples used (in the textbook) were ir- 
relevant to education. Like Mr. X. she too had noticed that students 
Had difficulty applyiij;! classroom- conqepts to practical problems. 
1. Defining the Session Goals ^ 

.Ms. Z decided to use the comparative phase of the REMAR problem 
(tasks 6 and 7) to provide practice in the application of the ideas presented 
in class. It was her hope that a problem-solving approach tp the data 
would also spark student curiosity thus making the entire course more mean- 
ingful and perhaps even less difficult. Procedurally, she began with a 
rough statement of goals which were refined and elaborated after she com- 
pleted steps (2) arid (3) which follow. For the reader's convenience these 
goals are stated here in their final fomr. The session goals were: 

a. Given a specific target population and a set of research hypotheses 

_ which relate changes in a dependent varia^e to various combinations 
of treatment, modifier, and control variables, each student will 
be able to design a study which provides a valid test of (the null 
'fonn of) each hypothesis, collect experimental data from the FEHR 
data generator, analyze the data using an appropriate statistical 
technique, interpret the educational implications. of the results, 
and, on the basi.s of that interpreted on, recommend the treatments 
to be used with various members of the target population. 

b. Student will achieve as well or better than students from previous 
class on the written examinations used last term. ^ 

c. Student complaints of irrelevant, overtheoretical , and over- 
difficult content will be reduced. 

2. Choosing Personnel for the Sfession 

Unlike the instructors In the two previous examples, Ms. .Z did not 
have access to either a course assistant or laboratory supervisors. After 
a careful examination of al! the supervisory tasks entailed, she decided 
that she could perform all three functions -instructional, jame manager, 
and consultant - herself. In fact, except for the bookkeeping involved 
in recording grants and check payments, the administration of a FEHR - PRAC- 
^ TIClli appeared very similar to the supervision of the regular laboratory 



problems she hc>J used previously. 

3. Familiarizing Personnel with the Data Generator 

Ms. Z completed each sample run provided in the general guidelines, 
and then re-ran several examples trying various combinations of the Card 
1 parameters. 

,4. Fami 11 arizing Personnel with the Practicum Process 

Ms. Z completed an entire practicum herself, followihy the Player's 
Instructions assiduously. After running through the practicum, she devised 
a series of increasingly complex factorial experiments always using treat- 
ment as one of the factors and various moderator variables (e.g.. sex, socio- 
economic status. IQ. initial math ability, etc.) as additional factors. She 
also experimented with a variety of sampling procedures to eliminate, equate, 
or balance out the effects wf variables she arbitrarily designated as control 
variable^ (e.g.. sex. race, fathers education, etc.). At the end of this 
procedure, she was confident that FEHR-PRACTICUM would serve her purposes. 
At this point she returned to the session goals (step 2) and revised them to 
their present form. 

5. Choosing a Place to Meet 

The classroom in which Ms. Z's regular class had been held was suitable 
for both largs-group meetings and for team work sessions. 

6. Selecting Materials for the Session 

The regular FEHR materials listed on page of this manual were ordered, 
with three copies o^ all materials marked ^h a double asterisk. Copies 
of the suggested references and marty of the auxiliary references were avail- 
able at the University Library. 

7. Choosing the Practicum Tasks 

It was obvious from the list of goals that a complete FEHR-PRACTICUM 
was unnecessary. Ms. Z decided thai she would supply an operational definitio 
of the problem, a listing of the target population of computationally dis- 
advantagid students and complete sunmary statistics (means and standard devia- 
tions for various subgroups of the population (e.g.. grouped by sex. grouped 
by socio-economic status, and the like). In addition, eac'.i team was to be 



180 



supplied with a punchcard copy of the survey data on each member of the 
target population. These cards could^then be physically sorted into various 
treatment groups and used to run the experimfent in the data generator. (When 
this is done, card 1 of each computer rUn must contain a 1 in column 5.) 
Thus the same subjects (cards) can be sampled in various ways to test sets 
of hypotheses implying different research designs. Two examples will suffice: 
a. Hypothesis . When the treatment groups are of equal intelligence, 
the AUTOMATH program will produce larger scores on the SAT computa- 
tion subtest than the regular class. 

Implied design . A one way randomized block (or possibly matched 
groups) design ds implied, with groups blocked by intelligence. 
0. Hypothesis . Other things being equal, subjects will, on the av- 
erage, gain mo;:e<orr>the SAT computation test with the AUROMATH pro- 
9.ams than in the rejjular class. However, this difference will 
be much more pronotfrfced for subjects of low intelligence than for 
those of average or high intelligence. 

Implied Design . A three-way factorial randomized groups design 
is simplified (treatment by intelligence by time), with repeated 
measures (pre and post) on. the time factor. Randomization is the 
only method of equating groups on all "other things". 
The complete listing of assigned practicum tasks appear on pages to 
. The general assignments given to the teams reflect the fact tha only 
Ms. Z will be available to players. Thus', students are required to budget 
their requests and to keep track of their expenditures, but they do not 
have to pay for requests with checks. (Nor does Ms. Z have to keep track of 
them.) 

8. Planning the Instructional Activities for the- Session 

As in example 2, each student (player) will be expected to finish a 
series of experiments defined by hypotheses supplied by Ms: Z. The hypotheses 
will be designed to lead students through progressively more complex designs. 
In each case the theory will be supplied in a class lecture before the as- 
signment is given. 

Although individual proposals and eports will be assigned, students will 
be encouraged to work in groups to develop processes. The individualized nature 
of their assigned subjects and variables pennits close consultation on process 
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CHECKLIST OP TASKS TO BE PERFORMED 

Since the work tb be accomplished under this RPP \ aries from loca- 
tion to locatioi\f your Game Manager has placed a ( ) before the 
particular items for which your team is responsible in the PRACTICUM- 
(If no items are checked^ see your Game Manager before proceeding.) 
h' detailed explanation of how to acccHaplish each task appears in the 
Instructions section on the page indicated at right. Tasks marked 
with an asterisk (*) are mandatory in every PRACTICUM. Before be- 
giniiing, each player should study, interrelate, and synthesize the 
information contained in the RFP, ibs various appendices, and the 
relevant paJrts.«of the Pair City Booklet « 

-' . ' " ^ 

Assigned . Instructions 

Tasks Description of Task , Page 

* (1) Each player should, submit an Orientation 1 
' Questionnaire to the Gamfe Manager. The 

specific q^estion to be answered are in 
the INSTRUCTIONS. . ' 

? 

> » (2) Form teams as directed by the Game Manager. 1 
The following assignments and requirements 
apply throughout this ' PRACTICUMf 

^Team salary: $ per dhy. 

costs are to be paid as*^ indicated below: • 

(a) ignore all costs 

(b) pay costs by check • 
(ci pay costs with computer receipts 

Requests for file searches, survey and treat- 
ments should be: ' 

(a) entered by the team via PROMPTS 
^^(b) Entered 1^ the team via punch cards 
_7(c) written on the appropiiate foAi and 
submitted to the Game Manager ^ 

, A Log of r:esearch activities: 

(a) to be kept by each player 
(bj to be kept by each team 

(c) no long necessary 

» (3) Operationalize the problem observing the fol- 3-5 
lowing restrictions: 

Literature: n o review necessary 

Information Bank only 

information Bank and regular 

library 

Definition: Problem definition provided 

by Game Manager 

c omplete problem definition 

form 
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Instructions 

'"'JiliJs Description of Task P^^^- 



(4) 



Determining Extent and Severity of the 5-7 
Problem 

_4.1 Submit a Request for a Planning 
Grant on the form provided 



Due Date: 

Maximum Grant $_ 



4.2 Run File Search and/or Survey 

Maximum sample size: 

Maximum number of . variables. 



(5) 



(6) 



( independent; dependent) 

•4.3 Analyze the data from 4.2 using 

appropriate statistical procedures 

4.4 Formal Problem Description (first 

draf*-) . To be presented for cri- 
tiquing by the other teams on 
(date) 

Funds are available for identifying a Pool ^-7 . 
of Research Subjects for^ Treatments 

Yes? ^No 

Empirical Evaluation of the Education ?-9 
Treatments 

^6.1 Submit a Research proposal to the 
Ganie Manager. The p oposal must 
meet these restrictions and re- 
quirements: 

Maximum number of variable." p^<r ob^ 
servation: 

Dependerit ; mdcjiendent : 

Total 

Maximuiti, number of observation l- 

times . 

Mal5^"" number of research subjects/ 

Date proposal must be submitted_ 
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The maximum project budget is: 

The proposal should contain each item 
checked below: 

(a) abstract of the proposal 

(b) introduction and overview of 

problem 
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Desci^iption of Task 



Instructions 
page 



^ c) a review of the related 

literature from: 

thU Information Bank only 

^_ boc h the Information Bank 

(d) a formal statement of the problem 

(e) a description of the procedures 

*' to be used / 

J (f) a schedule of activities / 

(g) an itemized budget 

The writing style for the proposal is 
checked below: 

(a) brief outline 

(b) detailed outline 

(c) formal narrative 

6.2 using an appropriate method, divide 
your sample into "treatment groups, 
observing^ the restrictions given in 
Task 3. Ruq the experiment on the 
PEHR Data Generator. Data will be 
output on: 

_(a) printed copy only 

(b) printed copy and punch cards 

(c) printed copy and the team's 

computer file 

6.3 Analyze and interpret the data fran 
Step^6.2 using appropriate statisti- 
cal procedures. Record restrictions 
(to be announced by the Game Manager) 
below: ^ 



6.4 Submit a final report to the Game Mana- 
ger. The report should contain the 
proposal items checked in 6.1 plus the^ 
items checked below: 

(h) the results of the field test 

(i) h summary of the statistical 

findings f^S^ 
(j) a discussion of results Culmi- 
nating in an educatuonaljdecision 
' _(k) a discussion of the\ limitations 
of the project 
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Assigned 
Tasics 



(7) 



Description of 1 oK 



)tnstructions 
page 



.(1) recorar.endations for future 

research ') 
(mV an appendix c/ort^ining the 
^^og(s)^ of activities 

The writing style of the final re- 
port should be: 

J (a) brief outline 

(b) detailed outline 

^(c) formal' narrative 



Final report due dates 



Attend^ consolidation Session (at a time 
to be announced by the Game Manager) to dis- 
cuss the streftgths and weaknesses of the 
strategies employed by your team, and^^ny 
.other teams which are attacking the same 
prpbleir 
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Mhllt $tni rtqulring an Individual application. The sets of variables to . 
bt used correspond closely to those described in experiment 2. 

9. Evaluating the Practicum Products 

£ach of the practicum products will be marked by Ms. Z as a regular 
laboratory assignment. It is planned .|p base a fairly large portion of the 
course grade (e.g., 75X?) on these products. Brief proposals and reports 
•re expected for hypc^thesei defining each of the following designs. 

a. cne way randomized groups 

b. *" one W4^ randomized blocks 

c. three way factorial 

d. factorial with repeated measures 

e. simple la;in square 

f. one way analysis of covariance 

g. factorial analysis of covariance 

. 10. Evaluatina the Instructional Effectiveness of the Practicum 

Ms. Z plans to evalual^e the ef fecti vsness of the practicum in three 

wQys: 

a. Ttfsts used in previous years will be administered to practicum 
students and their scores compared with those of previous classes. 

b. Students at the end of the course will be asked to fill out a 
questionnaire which invites critical «s:issment of the practicum 
projects. 

c. A record will be kept to compare the kinds of help required by 
students doing dissertations this year (no FEHR) and next (FEHR 
experience). 

f 
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